Image processing method and device applied to object attribute recognition, equipment, medium and product

Abstract

The embodiment of the invention provides an image processing method applied to object attribute recognition. The image processing method can be applied to the technical field of artificial intelligence. The image processing method applied to object attribute recognition comprises the following steps: acquiring a to-be-recognized sub-image of a target recognition image; determining an attribute identification tag of the to-be-identified sub-graph according to a matching relationship between the target feature value of the to-be-identified sub-graph and a template feature value set of a preset matching template; and determining corresponding object attribute information according to the attribute identification tag. The embodiment of the invention further provides an image processing device and equipment applied to object attribute recognition, a storage medium and a program product.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, specifically to the field of image processing technology, and more specifically to an image processing method, device, equipment, medium and product for object attribute recognition. Background Art

[0002] Artificial Intelligence (AI) is an important driving force for the new round of scientific and technological revolution and industrial transformation. It is a new key technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. As an important part of intelligent science, artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is a very broad science, including robots, speech recognition, image recognition, natural language processing, expert systems, machine learning and computer vision.

[0003] Computer Vision (CV) is a key branch of artificial intelligence, which aims to enable computers to understand and process visual information, that is, to "see". Computer vision involves many tasks, including image classification, target detection, image segmentation, pose estimation, object tracking, image generation, etc. Its ultimate goal is to enable machines to accurately understand visual data in different scenarios and make intelligent responses and decisions. The premise of understanding visual data is that the computer can accurately identify the objects in the acquired images, and the specific judgment standard can be the accuracy of the object's attribute recognition. However, existing object attribute recognition technology can usually only achieve relatively accurate recognition of a specific attribute feature, and cannot effectively support the accurate recognition of multiple or even all necessary attribute features. Summary of the invention

[0004] In view of at least one of the above problems, embodiments of the present invention aim to provide image processing methods, devices, equipment, media and products for object attribute recognition that can take into account the accurate recognition of multiple object attribute characteristics, thereby providing a technical solution that can be expanded to the accurate recognition of more than two object attribute characteristics at a relatively low cost, in order to achieve a higher level of intelligence.

[0005] An aspect of an embodiment of the present invention provides an image processing method for object attribute recognition, which includes: obtaining a sub-image to be recognized of a target recognition image; determining an attribute recognition label of the sub-image to be recognized based on a matching relationship between a target feature value of the sub-image to be recognized and a template feature value set of a preset matching template; and determining corresponding object attribute information based on the attribute recognition label.

[0006] According to an embodiment of the present invention, obtaining a sub-image to be recognized of a target recognition image includes: obtaining mask data information of the target recognition image; and obtaining the sub-image to be recognized of the target recognition image according to the mask data information.

[0007] According to an embodiment of the present invention, obtaining mask data information of a target recognition image includes: obtaining a category sub-image of the target recognition image that matches preset category information; and extracting mask data information of the category sub-image.

[0008] According to an embodiment of the present invention, in acquiring a sub-image to be recognized of a target recognition image according to mask data information, the method includes: generating a background image corresponding to the mask data information; and acquiring the sub-image to be recognized according to the background image.

[0009] According to one embodiment of the present invention, in determining the attribute identification label of the sub-image to be identified based on the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of a preset matching template, it includes: obtaining the similarity between the target feature value and each template feature value in the template feature value set; and determining the attribute identification label of the sub-image to be identified based on the similarity.

[0010] According to an embodiment of the present invention, determining the attribute identification label of the sub-graph to be identified according to the similarity includes: when the similarity is greater than or equal to a preset threshold, determining the attribute identification label of the template feature value corresponding to the similarity.

[0011] According to one embodiment of the present invention, determining corresponding object attribute information based on attribute identification tags includes: sorting the attribute identification tags according to similarity; determining the object attribute information corresponding to the attribute identification tags after the sorting; or determining the object attribute information corresponding to the attribute identification tag with the maximum similarity.

[0012] According to an embodiment of the present invention, the image processing method applied to object attribute recognition further includes: updating the corresponding attribute recognition tag according to the dynamic detection result of the preset matching template.

[0013] Another aspect of an embodiment of the present invention provides an image processing device for object attribute recognition, which includes a sub-image acquisition module, a label determination module and an information determination module. The sub-image acquisition module is used to acquire a sub-image to be recognized of a target recognition image; the label determination module is used to determine the attribute recognition label of the sub-image to be recognized based on the matching relationship between the target feature value of the sub-image to be recognized and the template feature value set of a preset matching template; and the information determination module is used to determine the corresponding object attribute information based on the attribute recognition label.

[0014] Another aspect of an embodiment of the present invention provides an electronic device, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image processing method applied to object attribute recognition.

[0015] Another aspect of an embodiment of the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned image processing method for object attribute recognition.

[0016] Another aspect of an embodiment of the present invention provides a computer program product, including a computer program, which implements the above-mentioned image processing method for object attribute recognition when executed by a processor.

[0017] The image processing method for object attribute recognition provided by the embodiment of the present invention can at least partially solve the problem that object attribute recognition cannot be expanded at low cost in the related art, and thus can achieve at least one of the following technical effects: The image processing method for object attribute recognition provided by the embodiment of the present invention can meet the high-precision recognition of more than two object attribute features by means of the image-text matching strategy defined by the matching relationship between the target feature value of the sub-image to be recognized and the template feature value set of the preset matching template, such as color, shape, texture, material, etc., which can be accurately recognized in parallel; at the same time, the background noise can be effectively reduced during the image processing process, greatly improving the accuracy of object attribute recognition; and the newly added attribute features in the preset matching template can also be directly recognized without the need for repetitive work such as iterative tuning, and the attribute categories can be arbitrarily expanded, with stronger versatility; further, combined with the hot loading mechanism, the online update of the newly added attribute categories can be realized at a low cost. Therefore, the above-mentioned image processing method of the embodiment of the present invention can meet the low-cost improvement of the recognition versatility of the entire system, with higher recognition accuracy, thereby greatly improving the intelligent level of recognition.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and illustrative and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which: Figure 1 A schematic diagram of an application scenario of an image processing method, apparatus, device, medium, and program product for object attribute recognition according to an embodiment of the present invention is shown; Figure 2 Schematically shows a flow chart of an image processing method applied to object attribute recognition according to an embodiment of the present invention; Figure 3A A schematic diagram of a scene diagram for obtaining a category subgraph of an image processing method for object attribute recognition according to an embodiment of the present invention is shown; Figure 3B A schematic diagram of a scene diagram for extracting mask data information of a category sub-graph of an image processing method for object attribute recognition according to an embodiment of the present invention is shown; Figure 4 A picture template diagram schematically shows an image processing method applied to object attribute recognition according to an embodiment of the present invention; Figure 5 A block diagram schematically shows a structure of an image processing device applied to object attribute recognition according to an embodiment of the present invention; and Figure 6 A block diagram of an electronic device suitable for implementing an image processing method for object attribute recognition according to an embodiment of the present invention is schematically shown.

[0020] The above-mentioned drawings are part of the specification of the embodiments of the present invention, which illustrate exemplary embodiments of the present invention. The attached drawings and the description of the specification are used together to illustrate the principles of the embodiments of the present invention. It should be understood that the above general description of the drawings and the following specific implementations are only exemplary and illustrative, and they cannot limit the scope of the present invention. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0022] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0023] The terms “first”, “second”, etc. used in the present invention do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0024] The directional terms used in the present invention, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0025] The words “include,” “including,” “have,” “contain,” etc. used in the present invention are open-ended terms, meaning including but not limited to.

[0026] The term "and / or" used in the present invention includes any or all combinations of the items mentioned.

[0027] Regarding the present invention, "plurality" includes "two" and "more than two"; regarding the present invention, "plurality of groups" includes "two groups" and "more than two groups".

[0028] The terms "substantially" and "approximately" used in the present invention are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0031] In the field of artificial intelligence technology, image recognition can identify various types of information on the acquired images, such as spatial information (space size, orientation, and even room type), object information (object category, size, position, orientation, color, and shape), and person information (2D skeleton points, 3D skeleton points, actions, and face ID, etc.). Among them, accurate identification of various attribute information of objects in images through computer vision technology has great practical value for industrial applications in various fields. For example, in the agricultural field, the growth state can be judged by identifying the color of the plant; in the logistics field, the loading method can be determined by the shape of the express delivery, etc.

[0032] At present, the mainstream object attribute recognition technology is mainly based on the RGB or HSV information of the target area in the image for classification tasks. The classification method is usually based on neural networks or template matching. However, these traditional methods require a lot of manpower and financial resources in the collection and annotation of training data and template design, resulting in high investment costs for object attribute recognition technology. In addition, the existing technology still lacks batch recognition of multi-attribute features. Each attribute usually needs to be implemented based on a single model. There is mutual interference between the recognition of different attributes. It is difficult to perform batch continuous recognition of specific attributes without affecting the recognition of other attributes. Moreover, the model output is usually relatively solidified, and the recognition ability is relatively fixed, and it is impossible to introduce the recognition of new attributes. In short, the main problem of current object attribute recognition technology is the lack of a universal object attribute recognition model, that is, there is very little ability to support the recognition of multiple attributes such as color, shape, texture, and material with high precision, and it is also difficult to expand the recognition of new attribute categories in a low-cost way.

[0033] Furthermore, for the recognition of an object attribute in an image, different recognition capabilities are required in different scenarios. For example, in a specific business scenario, it may be necessary to recognize red and yellow colors, while in other scenarios, it may be necessary to recognize pink and green colors. In traditional technology, for the same type of recognition tasks in different scenarios, from realizing red and yellow color recognition to pink and green color recognition, manpower is still needed to do similar repetitive work, such as collecting training data, cleaning data, data expansion, training models, and iterative tuning. This repetitive work greatly increases the R&D cost. Therefore, it is necessary to provide a more intelligent recognition solution that can quickly customize recognition requirements and update recognition requirements at any time. It can even be expanded to other object attribute recognition capabilities besides color, such as shape, material, texture, etc., which can greatly improve the intelligence level of the intelligent body.

[0034] In view of at least one of the technical problems existing in the above-mentioned prior art, the embodiments of the present invention aim to provide image processing methods, devices, equipment, media and products for object attribute recognition that can take into account the accurate recognition of multiple object attribute characteristics, thereby providing a technical solution that can quickly customize recognition requirements, update recognition requirements at any time, and even expand to the recognition capability of more other object attributes, thereby greatly improving the intelligence level of the intelligent body and expanding to the accurate recognition of more than two object attribute characteristics at a lower cost.

[0035] An aspect of an embodiment of the present invention provides an image processing method for object attribute recognition, which includes: obtaining a sub-image to be recognized of a target recognition image; determining an attribute recognition label of the sub-image to be recognized based on a matching relationship between a target feature value of the sub-image to be recognized and a template feature value set of a preset matching template; and determining corresponding object attribute information based on the attribute recognition label.

[0036] Figure 1 The application scenario diagram schematically shows an image processing method, apparatus, device, medium and program product for object attribute recognition according to an embodiment of the present invention.

[0037] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0038] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0039] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0040] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0041] It should be noted that the image processing method for object attribute recognition provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the image processing device for object attribute recognition provided in the embodiment of the present invention can generally be set in the server 105. The image processing method for object attribute recognition provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image processing device for object attribute recognition provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0043] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The image processing method applied to object attribute recognition in the disclosed embodiment is described in detail.

[0044] like Figure 2 As shown, one aspect of an embodiment of the present invention provides an image processing method for object attribute recognition, which includes operations S201 to S203.

[0045] In operation S201, a sub-image to be recognized of a target recognition image is obtained; In operation S202, an attribute identification label of the sub-image to be identified is determined according to a matching relationship between a target feature value of the sub-image to be identified and a template feature value set of a preset matching template; and In operation S203, corresponding object attribute information is determined according to the attribute identification tag.

[0046] Target recognition images are images with target recognition objects, which are usually related to the actual recognition needs of intelligent agents. For example, when a smart car is driving on the road, it needs to recognize objects (cars, people, animals, etc.) in the front, back, left, and right directions. The images captured by the camera on the car can be used as target recognition images, which contain target recognition objects. Only by accurately identifying the attribute characteristics of the target recognition object can the next action of the intelligent agent be executed.

[0047] The sub-image to be identified is a local processed image of the target identification object on the target identification image. The local processed image can exclude the identification of non-target identification objects (such as background images) as much as possible, improve the recognition accuracy of the corresponding object attributes of the target identification object, and also avoid unnecessary identification. For example, if only the person in the target identification image needs to be identified, only the local image of the framed person needs to be processed to generate the sub-image to be identified.

[0048] The target feature value is the feature value of the sub-image to be identified that is related to the target identification attribute corresponding to the target identification object. For example, if the target identification attribute is color, the target feature value can be a feature value related to color. In addition, the target identification attribute can also be multiple categories such as color, shape, material, etc. At this time, the target feature value can be a feature value related to multiple categories such as color, shape, material, etc., rather than a feature value of a single attribute.

[0049] The preset matching template is template data with corresponding attribute identification tags. These template data can define the corresponding feature value of each attribute feature, and can be specifically embodied in the form of text (such as text.txt) and / or graphics (such as picture.jpg).

[0050] For example, the text in txt format "a red object with white background" can be used as the attribute identification label of the color "red", which can be used to define the object attribute color as "red". Correspondingly, the picture in the picture format of "pink.jpg" can be used as the attribute identification picture of the color "pink", and the named text of the picture can be used as the attribute identification label of the color attribute. Among them, when these texts in txt format are embodied in the form of a list, they can constitute a text template list in txt format, and the text template list can be used as a component of the preset matching template. At the same time, these attribute pictures in jpg format named with text labels can be combined to constitute another component of the preset matching template.

[0051] Therefore, when the corresponding text feature values ​​or image feature values ​​of the text in the text template list or a certain attribute image are extracted separately, the corresponding template feature values ​​can be constructed to form a template feature value set (ie, feature value set).

[0052] The template feature value set may include all feature values ​​of text and image templates in the preset matching template, that is, it may include matching feature values ​​of various attributes. These matching feature values ​​can establish a mapping relationship with the target feature value of the sub-image to be identified based on the preset matching relationship. According to this type of mapping relationship, the template feature value that matches the target feature value can be found from the template feature value set, and the attribute identification label corresponding to the template feature value can correspond to the corresponding object attribute. Therefore, high-precision parallel recognition of multiple attribute categories can be achieved, and the process of each attribute recognition is carried out in parallel, which can improve the recognition efficiency while ensuring the recognition accuracy, thereby improving the recognition versatility.

[0053] The attribute identification label (Label) is the preset text or image information directly related to the attribute in the preset matching template, which can correspond to the feature value of the corresponding attribute. For example, the text in the txt format of "a red object with white background" can be used as the attribute identification label of the color "red", which can be used to define the object attribute color as "red", and the text has a corresponding text feature value B1 that can correspond to the attribute identification label of "red"; accordingly, the picture in the picture format of "pink.jpg" can be used as the attribute identification picture of the color "pink", the named text "pink.jpg" of the picture can be used as the attribute identification label of the color attribute, and the picture feature value B2 of the picture can correspond to the attribute identification label of "pink".

[0054] Therefore, the corresponding object attribute information can be directly defined through the attribute identification tag. The object attribute information can be the text name, characteristic value and other related information of the object attribute, which can be specifically expressed in the form of text or a combination of text and graphics. In addition, it can also include the encoding information of the corresponding attribute, which can be determined according to the actual attribute identification requirements.

[0055] The image processing method for object attribute recognition provided by the embodiment of the present invention can meet the parallel high-precision recognition of more than two object attribute features, such as color, shape, texture, material, etc., by means of the image-text matching strategy defined by the matching relationship between the target feature value of the sub-image to be recognized and the template feature value set of the preset matching template, and can realize accurate recognition in parallel; at the same time, it can effectively reduce the background noise in the image processing process, greatly improving the accuracy of object attribute recognition; and the newly added attribute features in the preset matching template can also be directly recognized without the need for repetitive work such as iterative tuning, and the attribute categories can be arbitrarily expanded, with stronger versatility; further, combined with the hot loading mechanism, it can realize the online update of the newly added attribute categories at a low cost. Therefore, the above-mentioned image processing method of the embodiment of the present invention can meet the low-cost improvement of the recognition versatility of the entire system, with higher recognition accuracy, thereby greatly improving the intelligent level of recognition.

[0056] like Figure 2-Figure 4 As shown, according to an embodiment of the present invention, in operation S201, obtaining a sub-image to be identified of a target recognition image includes: Obtain mask data information of the target recognition image; Obtain the sub-image to be identified of the target recognition image according to the mask data information.

[0057] The target recognition image can define corresponding mask values ​​(Mask) according to its corresponding attribute characteristics and other information based on the distribution position in the image. These mask values ​​are used to define the scope of local operation processing on the target recognition image, and realize further extraction, analysis and fusion of specific target areas. Mask data information is data information related to such mask values, which can define the color, pixel and position distribution of corresponding points on the image.

[0058] Therefore, the extraction operation of the sub-image to be identified can be realized more accurately through the mask data information, making the acquisition process of the identification sub-image faster and more effective, and eliminating more background factors. For example, after the target identification image is converted into mask data information, the distribution of the same mask value at different positions of the image can define a certain object. For example, the distribution area of ​​the mask value "1" in the entire target identification image can be defined as the distribution area of ​​the target identification object "car". Therefore, the local image where the target identification object "car" is located can be extracted in combination with the distribution area of ​​these value "1", and further processed to generate the corresponding sub-image to be identified.

[0059] It can be seen that the noise interference of non-target identification objects can be effectively eliminated, making the identification of target identification objects more accurate, thereby ensuring that the identification of target identification attributes of target identification objects is more accurate.

[0060] like Figure 2-Figure 4 As shown, according to one embodiment of the present invention, in acquiring mask data information of a target recognition image, the following is included: Obtaining a category sub-image of a target recognition image that matches preset category information; Extract the mask data information of the category sub-image.

[0061] The preset category information can be embodied in the form of text and / or code, and is specifically used as the basis for extracting the category sub-image of the target recognition image. It can define the target recognition objects and their corresponding attribute information that meet the user's recognition requirements. It can be implemented through user presets or as input data for the intelligent agent through instant operations.

[0062] The category sub-graph is a local image (i.e., Patch) generated by performing the target recognition object extraction operation on the target recognition image according to the preset category information. The specific extraction operation can be implemented using extraction tools such as the Open Set Object Detector (Grounding-DINO).

[0063] like Figure 3A As shown in the figure, the target recognition image is the left image, and the preset category information is "dog. stick.". Based on the information of "dog. stick.", the extraction operation can be performed through Grounding-DINO, and the corresponding three category sub-images as shown on the right side of the figure are generated according to the matching degree defined by the matching threshold, namely, the three local images of "yellow dog, black dog and stick". The details are not repeated here.

[0064] Furthermore, the mask data information of the above-mentioned category sub-image is further extracted, which can be realized by image segmentation tools such as Mobilesam. Figure 3B As shown in the figure, the category sub-image is a partial image of "car" with a partial background image generated by patch extraction. Mobilesam is used to further extract the mask data information of the category sub-image. It can achieve accurate extraction only for the image distribution area (blue area) where "car" is located and generate the corresponding mask data information (ie Mask).

[0065] Through the extraction of category sub-images and mask data information, the original background noise of the image can be effectively reduced, and high-precision segmentation of the background image can be achieved, thereby ensuring that all effective information corresponding to the target identification object in the category sub-image can be accurately obtained.

[0066] like Figure 2-Figure 4 As shown, according to one embodiment of the present invention, in obtaining a sub-image to be identified of a target recognition image according to mask data information, the following is included: generating a background image corresponding to the mask data information; Get the sub-image to be identified according to the background image.

[0067] According to the mask data information extracted from the category sub-image, a pure color image of the same size can be constructed as the background image corresponding to the above mask data information. The mask data information can define the size information of the original image occupied by the extracted target recognition object, so a pure color image can be directly generated as the background image based on the size information.

[0068] Furthermore, by using the difference between the foreground information of the category sub-image and the background information of the background image defined by the mask data information, a fusion process of the background image and the foreground image of the category sub-image is realized to form a sub-image to be identified with a pure color background.

[0069] Therefore, the sub-image to be identified can effectively eliminate the influence of background noise, and at the same time can contain all the effective information of the target identification object, making the subsequent identification of target identification attributes more accurate. Among them, the background image color can be matched according to the color of the foreground image of the category sub-image, and is not limited to a specific color, which will not be repeated here.

[0070] like Figure 2-Figure 4 As shown, according to an embodiment of the present invention, in operation S202, determining the attribute identification label of the sub-image to be identified according to the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template includes: Obtain the similarity between the target feature value and each template feature value in the template feature value set; The attribute identification label of the subgraph to be identified is determined according to the similarity.

[0071] By extracting the feature value of the sub-image to be identified, the corresponding multi-dimensional target feature value can be obtained. The specific extraction operation can be implemented by the corresponding feature extraction tool, such as Clip. Among them, the sub-image to be identified is an image of the target identification object with a pure background color relative to the original target identification image. For example, the 512-dimensional target feature value A of the sub-image to be identified can be extracted through Clip's image encoder (PictureEncoder).

[0072] Each template feature value of the template feature value set of the preset matching template is also a value of the dimension corresponding to the target feature value. For example, the template feature value set is [B1, B2, B3, ..., Bn], where each Bx value is a template feature value, and is also a value of the same dimension (512 dimensions) as the target feature value A.

[0073] Furthermore, the target feature value and each template feature value are matched, and specifically, the target feature value A can be matched with each template feature value Bx in the template feature value set [B1, B2, B3, ..., Bn] by calculating the cosine similarity, thereby determining the corresponding similarity. According to the magnitude of these similarities, the template feature value Bx that matches the target feature value A is determined, and then the corresponding attribute category label is determined according to the matched template feature value Bx.

[0074] Specifically, assume that the three attributes of color, material, and shape can be used as the user's target recognition attributes. After obtaining the corresponding target feature value of the target recognition object, all subdirectories colortemp (color), material temp (material), and shape temp (shape) under the current directory can be queried. For example, the subdirectory color temp satisfies: color_temp —red_blue.jpg —white_black.jpg —pink.jpg —text.txt A text template list can be defined in the text.txt file, and the text template list can list the following text information: Label1:a red object with white background Label2:a yellow object with white background Label3:a blue object with white background Label4:a green object with white background In this way, the above text information can be used as an attribute identification label to define four basic color attributes, namely red, yellow, blue and green. Feature value extraction is performed on each of them to generate four different template feature values ​​B1-B4.

[0075] In addition, red_blue.jpg, as a picture file, can define an image with both red and blue colors, which can be extracted separately as a template feature value B5. Among them, the picture template file can be used to define attributes that cannot be effectively defined by the text template, thereby greatly expanding the attribute matching range of the template. Specifically, Figure 4As shown in the figure, taking material as an example, there are 8 material images listed, including wood veneer, wood floor, stainless steel, metal mesh, cloth texture, exterior, marble, and Changhong glass gradient glass paint. In the directory of material temp, 8 jpg images need to be defined for each of the above materials, while the text list only needs to define simple and common materials.

[0076] Therefore, during the matching process, the template data of xxx.jpg and text.txt in all subdirectories can be counted respectively. Assuming that there are 10, 5 and 3 templates for color, material and shape attributes respectively, the target feature values ​​extracted from the target sub-image to be identified can be respectively calculated for similarity with the feature values ​​corresponding to these 10, 5 and 3 templates, and then the corresponding attribute identification labels can be matched according to the similarity calculation results. Finally, the final matching result can be obtained for each attribute according to the matched attribute identification labels, such as color is "red", material is "wooden", and shape is "square".

[0077] Therefore, with the help of the image-text matching strategy defined by the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template, more than two types of object attribute features can be recognized in parallel with high precision, such as color, shape, texture, material, etc., which can be accurately recognized in parallel. In addition, only some more general model tools can be used to realize the corresponding attribute recognition, thereby ensuring the low cost of the entire processing process while improving the level of intelligence, and at the same time making it have strong scene versatility and a wider range of application.

[0078] like Figure 2-Figure 4 As shown, according to an embodiment of the present invention, in determining the attribute identification tag of the subgraph to be identified according to the similarity, it includes: When the similarity is greater than or equal to a preset threshold, an attribute identification label of the template feature value corresponding to the similarity is determined.

[0079] As mentioned above, for the matching results of the similarity between the target feature value corresponding to the sub-image to be identified and each template feature value in the template feature value set, a preset threshold based on the similarity can be used to extract the corresponding template feature values ​​in the matching results that are greater than or equal to the preset threshold, thereby obtaining the attribute identification label of the corresponding template based on these template feature values.

[0080] For example, after performing similarity calculation and matching between the template feature values ​​Bx of the 10 templates in color temp and the target feature value A, it is found that the similarity values ​​corresponding to the two colors "red" and "orange" in the corresponding attribute identification labels exceed the preset threshold, then the attribute identification labels corresponding to the two colors "red" and "orange" can be determined, such as "a red object with whitebackground" and "a orange object with white background".

[0081] Therefore, the attribute identification label that is closest to the target identification attribute of the target identification object can be accurately obtained.

[0082] like Figure 2-Figure 4 As shown, according to an embodiment of the present invention, in operation S203, determining corresponding object attribute information according to the attribute identification tag includes: Sorting the attribute identification tags according to the similarity; determining the object attribute information corresponding to the attribute identification tags after the sorting; or Determine the object attribute information corresponding to the attribute recognition label with the maximum similarity.

[0083] The above-mentioned matching strategy of the embodiment of the present invention can select the corresponding template feature value whose similarity is greater than or equal to the preset threshold value and extract it according to the similarity calculation result of the target feature value and each template feature value in the matching relationship. Among them, the template feature value Bx can be understood as the feature similarity of the corresponding template, and the corresponding cosine similarity can be obtained by matching and calculating it with the corresponding target feature value A. For B1-Bx corresponding to the cosine similarity exceeding the preset threshold of similarity, the label (label1-labelx) of the template corresponding to the template feature value B1-Bx can be directly extracted as the attribute identification label, and each extracted attribute identification label corresponds to a different cosine similarity. Therefore, according to the size of these cosine similarities, the above-mentioned attribute identification labels can be sorted. Then, the corresponding object attribute information can be determined in sequence for these sorted attribute identification labels.

[0084] Similarly, without considering the above-mentioned preset threshold of similarity, all the calculated attribute identification tags can be sorted for all the matching results of cosine similarity, and the corresponding object attribute information can be determined in sequence. Among them, the object attribute information can include the names, features and even codes of all the sorted object attributes, such as "color, red, red1 (code)". Further, the attribute identification tag with the greatest similarity in the matching result can be directly queried, and the corresponding object attribute information can be output. For example, the target feature value of the sub-image to be identified can be calculated and matched with the n text information in the text template list of text.txt in the text template color_temp and the corresponding template feature values ​​of multiple jpg of the picture template, and finally an attribute identification tag with the largest cosine similarity can be selected as the color tag. Correspondingly, the identification of material (material_temp) and shape (shape_temp) can also be matched in parallel, and the details will not be repeated. Therefore, different object attribute information can be pushed and displayed to users in different ways, so as to give users a more intelligent use experience.

[0085] Therefore, it is possible to realize parallel matching and recognition of multiple attributes at the same time, and effectively avoid the situation in the prior art where the data preparation work needs to be repeated independently for each attribute recognition, which greatly improves the processing accuracy and efficiency of the image processing process, and also greatly improves the intelligence level of the intelligent body.

[0086] like Figure 2-Figure 4 As shown, according to an embodiment of the present invention, the image processing method applied to object attribute recognition further includes: updating the corresponding attribute recognition label according to the dynamic detection result of the preset matching template.

[0087] During the operation of the agent, the file content under a specific preset matching template directory can be parsed when the system starts. As mentioned above, each subdirectory represents the identification of a type of attribute, and each attribute will be matched with a category of corresponding similarity through the above matching strategy.

[0088] During the operation of the system, each subdirectory of the preset matching template can be monitored in real time. Once it is found that the folder or file of the corresponding subdirectory has been added, deleted, modified or checked (for example, when the user updates the file content or even the file structure of the preset matching template), the corresponding dynamic detection result will be generated. The dynamic detection result is the dynamic instruction generated by the detection of the modification operation of the preset matching template. According to the dynamic instruction, an automatic loading mechanism such as hot loading can be automatically triggered, and the system will reload the modified and updated preset matching template. For example, if a line of a black object with white background is added to text.txt in the color directory color temp, the system can re-execute the loading process of the preset matching template through an automatic loading mechanism such as hot loading, thereby completing the recognition of the new attribute (color black) by itself. Specifically, the loading target can be the text or picture template of the corresponding directory after modification, or it can be the automatic record of the newly added attribute template for the new folder, such as adding a matching template of "texture" based on the matching template of "color, material and shape", which can also be achieved through the above loading mechanism.

[0089] Therefore, the above method of the embodiment of the present invention can utilize the automatic loading mechanism to perform online updates of the corresponding templates, so that the attribute categories required to be identified can be arbitrarily expanded, including the addition of text templates and image templates, without affecting the processing of the current process, thereby achieving a low-cost operation effect.

[0090] In summary, the image processing method for object attribute recognition provided by the embodiment of the present invention can meet the parallel high-precision recognition of more than two object attribute features, such as color, shape, texture, material, etc., by means of the image-text matching strategy defined by the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template, and can realize accurate recognition in parallel; at the same time, the background noise can be effectively reduced during the image processing process, greatly improving the accuracy of object attribute recognition; and the newly added attribute features in the preset matching template can also be directly recognized without the need for repetitive work such as iterative tuning, and the attribute categories can be arbitrarily expanded, with stronger versatility; further, combined with the hot loading mechanism, the online update of the newly added attribute categories can be realized at a low cost. Therefore, the above-mentioned image processing method of the embodiment of the present invention can meet the low-cost improvement of the recognition versatility of the entire system, with higher recognition accuracy, thereby greatly improving the intelligent level of recognition.

[0091] Based on the above-mentioned image processing method for object attribute recognition, the present invention also provides an image processing device for object attribute recognition. Figure 5 The device is described in detail.

[0092] Figure 5 The structure block diagram of an image processing device applied to object attribute recognition according to an embodiment of the present invention is schematically shown.

[0093] like Figure 5 As shown, the image processing device 500 applied to object attribute recognition in this embodiment includes a sub-image acquisition module 510 , a label determination module 520 and an information determination module 530 .

[0094] The sub-image acquisition module 510 is used to acquire a sub-image to be recognized of the target recognition image. In one embodiment, the sub-image acquisition module 510 can be used to perform the operation S201 described above, which will not be described in detail here.

[0095] The label determination module 520 is used to determine the attribute identification label of the sub-image to be identified according to the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template. In one embodiment, the label determination module 520 can be used to perform the operation S202 described above, which will not be repeated here.

[0096] The information determination module 530 is used to determine the corresponding object attribute information according to the attribute identification tag. In one embodiment, the information determination module 530 can be used to perform the operation S203 described above, which will not be described in detail here.

[0097] According to an embodiment of the present invention, any multiple modules of the sub-graph acquisition module 510, the label determination module 520 and the information determination module 530 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the sub-graph acquisition module 510, the label determination module 520 and the information determination module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the sub-graph acquisition module 510, the label determination module 520 and the information determination module 530 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0098] Figure 6A block diagram of an electronic device suitable for implementing an image processing method for object attribute recognition according to an embodiment of the present invention is schematically shown.

[0099] The electronic device provided by an embodiment of the present invention includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image processing method applied to object attribute recognition.

[0100] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0101] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0102] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0103] The present invention also provides a computer-readable storage medium on which executable instructions are stored. When the instructions are executed by a processor, the processor executes the above-mentioned image processing method for object attribute recognition.

[0104] The computer-readable storage medium may be included in the device / apparatus / system described in the above embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0105] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0106] An embodiment of the present invention also includes a computer program product, which includes a computer program. When the computer program is executed by a processor, the image processing method for object attribute recognition is implemented.

[0107] The computer program includes program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0108] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0109] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0110] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0111] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0114] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. An image processing method for object attribute recognition, characterized in that: include: Obtain a sub-image to be identified of the target recognition image; Determining an attribute identification label of the sub-image to be identified according to a matching relationship between a target feature value of the sub-image to be identified and a template feature value set of a preset matching template; as well as Corresponding object attribute information is determined according to the attribute identification tag.

2. The method according to claim 1, characterized in that In the sub-image to be identified of the target identification image, the following is included: Obtaining mask data information of the target recognition image; A sub-image to be identified of the target recognition image is obtained according to the mask data information.

3. The method according to claim 2, characterized in that The mask data information for obtaining the target recognition image includes: Acquire a category sub-image of the target recognition image that matches preset category information; Extract the mask data information of the category sub-image.

4. The method according to claim 2, characterized in that: In the step of obtaining the sub-image to be identified of the target recognition image according to the mask data information, it includes: generating a background image corresponding to the mask data information; The sub-image to be identified is obtained according to the background image.

5. The method according to claim 1, characterized in that In the step of determining the attribute identification label of the sub-image to be identified according to the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template, the step includes: Obtaining the similarity between the target feature value and each template feature value in the template feature value set; An attribute identification label of the subgraph to be identified is determined according to the similarity.

6. The method according to claim 5, characterized in that In determining the attribute identification tag of the subgraph to be identified according to the similarity, including: When the similarity is greater than or equal to a preset threshold, an attribute identification label of the template feature value corresponding to the similarity is determined.

7. The method according to claim 6, characterized in that In the determining corresponding object attribute information according to the attribute identification tag, the following steps are included: Sorting the attribute identification tags according to the similarity; determining object attribute information corresponding to the attribute identification tags after the sorting; or Determine the object attribute information corresponding to the attribute identification tag with the maximum similarity.

8. The method according to claim 1, characterized in that Also includes: The corresponding attribute identification tag is updated according to the dynamic detection result of the preset matching template.

9. An image processing device for object attribute recognition, characterized in that: include: A sub-image acquisition module is used to acquire a sub-image to be identified of a target recognition image; A label determination module, used to determine the attribute identification label of the sub-image to be identified according to the matching relationship between the target feature value of the sub-image to be identified and the template feature value set of the preset matching template; as well as The information determination module is used to determine the corresponding object attribute information according to the attribute identification tag.

10. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images