Object recognition method and device, program product and electronic equipment

Abstract

The invention provides an object recognition method and device, a program product and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: determining a to-be-recognized image; intercepting a regional image of an object in the to-be-recognized image, and carrying out scaling processing on the regional image to obtain a plurality of scaling images; determining a target mask image according to the plurality of scaling images; and enabling the target mask image to correspond to the regional image, obtaining a target image only comprising the object, and carrying out recognition processing on the object based on the target image to obtain a recognition result. According to the method and the device, the target mask image determined by the appropriate multi-scale scaling image is adopted, accurate segmentation of the object is realized, the accuracy of obtaining the target image only comprising the object is improved, and feature extraction matching is performed based on the target image, so that the background influence can be greatly reduced, and the object re-identification accuracy is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an object recognition method, device, program product, and electronic device. Background Art

[0002] At present, in the fields of security monitoring, smart retail, human-computer interaction, etc., in order to solve the problem of "pedestrian identity association across perspectives and scenarios", a pedestrian re-identification (Person Re-Identification, ReID for short) technology system has been proposed.

[0003] In existing pedestrian re-identification technology systems, pedestrian detection processing is generally performed on the input image to obtain the pedestrian's external detection frame, and the pedestrian's external detection frame is output to the pedestrian re-identification model, thereby outputting a similarity sequence of pedestrians in the image. Summary of the Invention

[0004] However, since the external detection frame of the pedestrian also includes the cluttered background area outside the pedestrian, when the external detection frame is input into the pedestrian re-identification model for processing, the accuracy of the re-identification obtained is low due to the background part in the external detection frame.

[0005] The present disclosure provides an object recognition method, an object recognition device, a computer program product, and an electronic device to improve the accuracy of object recognition and further enhance the accuracy of object re-recognition.

[0006] According to a first aspect of the present disclosure, there is provided an object recognition method, the method comprising:

[0007] Determine an image to be recognized;

[0008] intercepting a regional image of the object in the image to be recognized, and performing scaling processing on the regional image to obtain a plurality of scaled images;

[0009] determining a target mask image according to the multiple scaled images;

[0010] The target mask image is mapped to the region image to obtain a target image that only includes the object, and recognition processing is performed on the object based on the target image to obtain a recognition result.

[0011] In a possible implementation, scaling the regional image to obtain a plurality of scaled images includes:

[0012] Determine multiple preset resolutions;

[0013] The original resolution of the regional image is scaled to the multiple preset resolutions respectively to obtain multiple scaled images.

[0014] In a possible implementation, determining a target mask image according to the multiple scaled images includes:

[0015] Inputting the plurality of scaled images into a segmentation model to obtain mask sequences corresponding to the plurality of scaled images, each of the mask sequences including a plurality of mask images and confidence information of each mask image;

[0016] An object mask image in each of the mask sequences is determined, and the target mask image is determined based on the obtained multiple object mask images.

[0017] In a possible implementation, determining the object mask image in each mask sequence includes:

[0018] For each mask sequence, do the following:

[0019] Determine pixel aspect ratio information of each mask image in the mask sequence;

[0020] Determining a first candidate set according to pixel aspect ratio information of each mask image and a preset ratio threshold;

[0021] Performing image-text matching processing on each mask image in the first candidate set to determine an object mask image in the mask sequence.

[0022] In a possible implementation, determining the target mask image according to the obtained multiple object mask images includes:

[0023] performing scaling processing on the plurality of object mask images to obtain a plurality of adjusted object mask images;

[0024] The target mask image is determined according to the adjusted multiple object mask images.

[0025] In a possible implementation, determining the target mask image according to the adjusted multiple object mask images includes:

[0026] Creating a blank image of the same size as the region image;

[0027] determining comprehensive pixel position information of the object based on the pixel position information of the object in the adjusted plurality of object mask images and confidence information corresponding to the object mask images;

[0028] Drawing is performed in the blank image according to the integrated pixel position information to determine the target mask image.

[0029] In a possible implementation, mapping the target mask image to the region image to obtain a target image including only the object includes:

[0030] The target mask image is mapped to the region image, pixels in the region image corresponding to the position of the object in the mask image are set as original pixels, and pixels in positions other than the object in the mask image are set as black pixels, so as to obtain a target image including only the object.

[0031] According to a second aspect of the present disclosure, there is provided an object recognition device, the device comprising:

[0032] A receiving unit, configured to determine an image to be recognized;

[0033] an obtaining unit, configured to intercept a regional image of an object in the image to be identified, and perform scaling processing on the regional image to obtain a plurality of scaled images;

[0034] a determining unit, configured to determine a target mask image based on the multiple scaled images;

[0035] The recognition unit is configured to correspond the target mask image to the region image to obtain a target image including only the object, and perform recognition processing on the object based on the target image to obtain a recognition result.

[0036] In a possible implementation manner, the obtaining unit is configured to:

[0037] Determine multiple preset resolutions;

[0038] The original resolution of the regional image is scaled to the multiple preset resolutions respectively to obtain multiple scaled images.

[0039] In a possible implementation, the determining unit is configured to:

[0040] Inputting the plurality of scaled images into a segmentation model to obtain mask sequences corresponding to the plurality of scaled images, each of the mask sequences including a plurality of mask images and confidence information of each mask image;

[0041] An object mask image in each of the mask sequences is determined, and the target mask image is determined based on the obtained multiple object mask images.

[0042] In a possible implementation, the determining unit is configured to:

[0043] For each mask sequence, do the following:

[0044] Determine pixel aspect ratio information of each mask image in the mask sequence;

[0045] Determining a first candidate set according to pixel aspect ratio information of each mask image and a preset ratio threshold;

[0046] Performing image-text matching processing on each mask image in the first candidate set to determine an object mask image in the mask sequence.

[0047] In a possible implementation, the determining unit is configured to:

[0048] performing scaling processing on the plurality of object mask images to obtain a plurality of adjusted object mask images;

[0049] The target mask image is determined according to the adjusted multiple object mask images.

[0050] In a possible implementation, the determining unit is configured to:

[0051] Creating a blank image of the same size as the region image;

[0052] determining comprehensive pixel position information of the object based on the pixel position information of the object in the adjusted plurality of object mask images and confidence information corresponding to the object mask images;

[0053] Drawing is performed in the blank image according to the integrated pixel position information to determine the target mask image.

[0054] In a possible implementation, the identification unit is configured to:

[0055] The target mask image is mapped to the region image, pixels in the region image corresponding to the position of the object in the mask image are set as original pixels, and pixels in positions other than the object in the mask image are set as black pixels, so as to obtain a target image including only the object.

[0056] According to a third aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method of the first aspect and possible implementations thereof are implemented.

[0057] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method of the above-mentioned first aspect and its possible implementation methods by executing the executable instructions.

[0058] The technical solution disclosed in this disclosure has the following beneficial effects:

[0059] In an embodiment of the present disclosure, an image to be identified can be determined, and then a regional image of an object in the image to be identified can be intercepted, and the regional image can be scaled to obtain multiple scaled images, thereby determining a target mask image based on the multiple scaled images. Furthermore, the mask image can be mapped to the regional image to obtain a target image that only includes the object, and the object can be identified based on the target image to obtain a recognition result. It can be seen that the present disclosure uses a plurality of appropriate scaled images of different scales to achieve accurate segmentation of the object, improve the accuracy of obtaining a target image that only includes the object, and, based on the target image, the object is identified to obtain a recognition result, that is, the influence of background factors on the object re-identification is greatly reduced, thereby improving the recognition accuracy of the object re-identification.

[0060] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present disclosure. The purposes and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments of the present disclosure. Obviously, the drawings introduced below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0062] Figure 1 A schematic diagram of an application scenario in this exemplary embodiment is shown;

[0063] Figure 2 A schematic flow chart showing an object recognition method in this exemplary embodiment is shown;

[0064] Figure 3 A schematic diagram showing a process of determining a target mask image in this exemplary embodiment is shown;

[0065] Figure 4 A schematic diagram showing a pedestrian recognition method in this exemplary embodiment;

[0066] Figure 5 A schematic structural diagram of an object recognition device according to this exemplary embodiment is shown;

[0067] Figure 6 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure. In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0069] The terms "comprises" and "comprising" and any variations thereof in the specification and claims of this disclosure are intended to cover non-exclusive protection. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0070] In the embodiments of the present disclosure, one or more, "many" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or plural.

[0071] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order, sequence, size and priority. For example, the first scaled image and the second scaled image in the embodiment of the present disclosure are only used to distinguish different scaled images. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0072] The following describes exemplary embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are schematic diagrams of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, or in hardware modules or integrated circuits, or in networks, processors or microcontrollers. The embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. The features, structures or characteristics described in the present disclosure can be combined in one or more embodiments in any suitable manner. In the description below, many specific details are provided to provide a full description of the embodiments of the present disclosure. However, those skilled in the art will appreciate that one or more specific details may be omitted when implementing the technical solutions of the present disclosure, or that other methods, components, devices, steps, etc. may be used to replace one or more specific details.

[0073] It should be noted that in the embodiments of this disclosure, certain software, components, models, and other existing industry solutions may be mentioned. These should be considered exemplary and are intended solely to illustrate the feasibility of implementing the technical solutions of this disclosure. This does not mean that the applicant has or will necessarily use such solutions. The collection, dissemination, and use of data in this technical solution comply with relevant national laws and regulations.

[0074] As mentioned above, since the external detection frame of the pedestrian also includes the cluttered background area outside the pedestrian, when the external detection frame is input into the pedestrian re-identification model for processing, the accuracy of the re-identification obtained is low due to the background part in the external detection frame.

[0075] Therefore, how to improve the recognition accuracy of pedestrian re-identification has become an urgent problem to be solved.

[0076] In view of this, an exemplary embodiment of the present disclosure provides an object recognition method, through which an image to be recognized can be determined, and then a regional image of the object in the image to be recognized can be intercepted, and the regional image can be scaled to obtain multiple scaled images, thereby determining a target mask image based on the multiple scaled images. Furthermore, the mask image can be mapped to the regional image to obtain a target image that only includes the object, and the object can be recognized based on the target image to obtain a recognition result. It can be seen that the present disclosure uses a plurality of appropriate scaled images of different scales to achieve accurate segmentation of the object, improve the accuracy of obtaining a target image that only includes the object, and, based on the target image, the object is recognized to obtain a recognition result, that is, the influence of background factors on the object re-recognition is greatly reduced, thereby improving the recognition accuracy of the object re-recognition.

[0077] To better understand the technical solutions provided by the embodiments of the present disclosure, the following briefly introduces the application scenarios to which the technical solutions provided by the embodiments of the present disclosure are applicable. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present disclosure and are not limiting. In specific implementations, the technical solutions provided by the embodiments of the present disclosure can be flexibly applied according to actual needs.

[0078] In the embodiments of the present disclosure, object recognition technology and pedestrian re-identification technology can be used in application scenarios such as autonomous driving and assisted driving, intelligent traffic management, intelligent security systems, crowd monitoring for large-scale events, disaster relief, digital operation of offline stores, and targeted advertising, but this is not limited in the embodiments of the present disclosure.

[0079] See also Figure 1 As shown, Figure 1 This is an application scenario to which the technical solutions of the embodiments of the present disclosure can be applied. In this scenario diagram, a terminal device 110 and an electronic device 120 are included. Specifically, the terminal device 110 and the electronic device 120 can be directly or indirectly connected in communication via one or more networks 130. Optionally, the application scenario may also include other electronic devices, which can, for example, perform linked processing based on the output results of the electronic device 120, but this is not limited in the embodiments of the present disclosure.

[0080] In an embodiment of the present disclosure, a user can send an image to be identified based on the terminal device 110, and then the electronic device 120 can determine the image to be identified, and then intercept the regional image of the object in the image to be identified, and scale the regional image to obtain multiple scaled images; based on the multiple scaled images, determine the target mask image; correspond the mask image to the regional image to obtain a target image that only includes the object, and perform identification processing on the object based on the target image to obtain an identification result.

[0081] In an embodiment of the present disclosure, a user can send a video to be recognized via terminal device 110. Electronic device 120 then receives the video and separates it into separate image sequences, thereby determining the image to be recognized. The electronic device 120 then uses each image in the image sequence as an image to be recognized, thereby determining the image to be recognized. A regional image of the object in the image to be recognized is captured and scaled to obtain multiple scaled images. A target mask image is determined based on the multiple scaled images. The mask image is mapped to the regional image to obtain a target image containing only the object. The object is then recognized based on the target image to obtain a recognition result.

[0082] In the embodiments of the present disclosure, Figure 1The terminal device 110 can be a mobile phone, a tablet computer (PAD), a personal computer (PC), a smart TV, a smart watch, a smart speaker, a smart car device and a wearable device, but is not limited thereto. These devices can have the function of supporting web navigation and website interaction.

[0083] In the embodiment of the present disclosure, the electronic device 120 can be a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited to these.

[0084] Of course, the method provided in the embodiment of the present disclosure is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, such as an application scenario where only the electronic device 120 implements the object recognition method, which is not limited in the embodiments of the present disclosure.

[0085] To further illustrate the technical solutions provided by the embodiments of the present disclosure, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present disclosure provide the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present disclosure. In the actual processing process or when the device is executed, the method may be executed in the order of the methods shown in the embodiments or drawings or in parallel.

[0086] See Figure 2 , Figure 2 An object recognition method provided by an embodiment of the present invention is provided by an electronic device, the electronic device being, for example, Figure 1 The electronic device 120 is not limited in the embodiment of the present invention.

[0087] Step 201: Determine an image to be recognized.

[0088] In the embodiment of the present disclosure, the electronic device can receive an image to be identified sent by a terminal device to determine the image to be identified, or it can receive a video sent by the terminal device, and then split the video to obtain an image sequence, thereby using each image in the image sequence as an image to be identified. Of course, the electronic device can also determine the image to be identified on its own, and this is not limited in the embodiment of the present disclosure.

[0089] Step 202: intercepting a region image of the object in the image to be recognized, and performing scaling processing on the region image to obtain a plurality of scaled images.

[0090] In an embodiment of the present disclosure, an electronic device can perform object detection processing on an image to be recognized, thereby capturing a regional image of the object in the image to be recognized. For example, the electronic device can perform pedestrian detection processing on the image to be recognized, thereby capturing a regional image of the pedestrian in the image to be recognized. In other words, the regional image is an image of the pedestrian detection frame area in the image to be recognized that includes the pedestrian. It should be noted that, considering that there may be multiple pedestrians in the image to be recognized, the object recognition scheme in the present disclosure is described below using a single pedestrian as an example.

[0091] In the disclosed embodiments, the resolution of pedestrian areas in surveillance videos varies greatly in real-world scenarios, with pixels ranging from 100*100 to 1204*1204 or even larger. The accuracy of the segmentation model is affected by the target resolution. High target resolutions may exceed the receptive field, resulting in incomplete segmentation; low target resolutions may lack detailed information, leading to inaccurate segmentation. Therefore, the disclosed embodiments propose combining the segmentation characteristics of different resolutions and using a multi-scale fusion method to generate and determine pedestrian mask images.

[0092] In the embodiments of the present disclosure, multiple preset resolutions may be predetermined, and then the original resolution of the regional image may be scaled to each of the multiple preset resolutions to obtain multiple scaled images. The number of the multiple preset resolutions may be determined based on actual implementation and is not limited in the embodiments of the present disclosure.

[0093] Optionally, five resolutions are selected, so that the original resolution of the regional image can be adaptively scaled to the longest side of 128, 256, 512, and 1204. That is, the resolution of the regional image is adaptively scaled proportionally so that the longest side (width or height, whichever is larger) is adjusted to 128 pixels, 256 pixels, 512 pixels, and 1024 pixels, respectively. The aspect ratio is maintained during the scaling process, and the short side size is automatically calculated based on the scaling ratio of the longest side, thereby obtaining four scaled images, namely the first scaled image, the second scaled image, the third scaled image, and the fourth scaled image. For example, if the original resolution is 800×600 (the longest side is 800), when it is scaled to the longest side of 512, the new width = 512, the new height = 600×(512 / 800) = 384, and the final size of the scaled image is 512×384.

[0094] Step 203: Determine a target mask image according to the multiple scaled images.

[0095] In the embodiments of the present disclosure, see Figure 3 As shown, the electronic device may determine the target mask image by adopting but not limited to the following steps, including:

[0096] Step 301: Input a plurality of scaled images into a segmentation model to obtain mask sequences corresponding to the plurality of scaled images, each mask sequence including a plurality of mask images and confidence information of each mask image.

[0097] In the embodiments of the present disclosure, the segmentation model is any model that can be used to segment objects in an image, for example, it can be the Segment Anything Model (SAM), SAM2 and other variant image segmentation models based on SAM, which are not limited in the embodiments of the present disclosure.

[0098] In the embodiment of the present disclosure, the obtained multiple scaled images may be input into a SAM model, thereby obtaining mask sequences corresponding to the multiple scaled images.

[0099] For example, by inputting the regional image, the first scaled image, the second scaled image, the third scaled image and the fourth scaled image into the SAM model respectively, mask sequence 1, mask sequence 2, mask sequence 3, mask sequence 4 and mask sequence 5, as well as the confidence level (Con) of each mask image in each mask sequence can be obtained.

[0100] In the disclosed embodiment, the segmentation model for the scene of pedestrian segmentation in the pedestrian detection frame may segment only the upper body or head of the pedestrian, only the lower body of the pedestrian, the hat of the pedestrian, the entire pedestrian, and even objects in the background area. In other words, a mask sequence includes multiple mask images, such as mask image 1 including the upper body of the pedestrian, mask image 2 including the lower body of the pedestrian, mask image 3 including the hat of the pedestrian, mask image 4 including the entire pedestrian, and mask image 5 including trees in the background area.

[0101] Step 302: Determine the object mask image in each mask sequence, and determine the target mask image based on the obtained multiple object mask images.

[0102] In the disclosed embodiment, for each mask sequence, the following operations are performed: determining the pixel aspect ratio of each mask image in the mask sequence; determining a first candidate set based on the pixel aspect ratio of each mask image and a preset ratio threshold; and performing image-text matching on each mask image in the first candidate set to determine the object mask image in the mask sequence.

[0103] In the embodiment of the present disclosure, the pixel aspect ratio information of the mask image can be understood as the ratio of the maximum pixel aspect ratio of the mask area to the pixel aspect ratio of the image. The preset threshold can be 0.9, or 0.8, etc., which is not limited in the embodiment of the present disclosure.

[0104] In the disclosed embodiment, after determining the pixel aspect ratio information for each mask image in the mask sequence, the pixel aspect ratio information for each mask image can be compared with a preset ratio threshold. Mask images whose maximum pixel aspect ratio of the masked area relative to the pixel aspect ratio of the image is greater than the preset ratio threshold are selected as mask images in the first candidate set. In this way, a first candidate set of mask images that excludes mask images that include partial human figures and smaller objects in the background can be obtained.

[0105] For example, assuming that mask sequence 2 includes 5 mask images, namely mask image 1, mask image 2, mask image 3, mask image 4 and mask image 5, and assuming that the preset ratio threshold is that the ratio of the maximum pixel width and height of the mask area relative to the pixel width and height of the image is greater than 0.9, and among the 10 mask images, the pixel width and height ratio information of mask image 1 and mask image 4 is greater than 0.9, then it is determined that the first candidate set includes mask image 1 and mask image 4.

[0106] In the embodiment of the present disclosure, after obtaining the first candidate set, the first candidate set may be screened. Specifically, a pattern-text matching process may be performed on each mask image in the first candidate set to determine the object mask image in the mask sequence.

[0107] Optionally, the image-text matching process can be based on a contrastive language-image pretraining (CLIP) model, that is, by mapping the image and text into the same vector space through contrastive learning, so that the machine can understand the semantic relationship between the image content and the text description.

[0108] For example, assuming the text is "This is a complete image of a person" and the image is a mask image in the first candidate set, then "This is a complete image of a person" and the mask image in the first candidate set can be input into the CLIP model to obtain the association score information of each mask image in the first candidate set, and the mask image with the highest association score information in the first candidate set is used as the object mask image in the mask sequence, and the confidence information of the mask image with the highest association score information in the first candidate set is recorded.

[0109] In an embodiment of the present disclosure, after obtaining the object mask image corresponding to each mask sequence, the electronic device may perform scaling processing on the multiple object mask images to obtain multiple adjusted object mask images.

[0110] For example, the object mask images at the four scaled resolutions can all be scaled to their original sizes, that is, to images with the same size as the region image, so that the five scaled mask images can be marked as mask_orig, mask_128, mask_256, mask_512, and mask_1024, respectively.

[0111] In the embodiment of the present disclosure, after obtaining the multiple adjusted object mask images, a target mask image may be determined according to the multiple adjusted object mask images.

[0112] Optionally, the electronic device may create a blank image of the same size as the regional image; determine the comprehensive pixel position information of the object based on the pixel position information of the object in the adjusted multiple object mask images; and draw in the blank image according to the comprehensive pixel position information to determine the target mask image.

[0113] For example, create a blank image of the same size as the region image, traverse each pixel of each object mask image, and obtain the weighted result of the corresponding pixel in the blank image, that is, the aforementioned comprehensive pixel position information final (x,y) If final (x,y) If the value is greater than or equal to 0.6, the pixel is determined to be a pixel corresponding to the pedestrian area; otherwise, it is a background pixel. That is, the pixels with a value greater than the threshold (i.e., the aforementioned 0.6) in all the comprehensive pixel position information are filtered out as the pixels of the pedestrian, thereby finally filtering out the background part and obtaining the target mask image.

[0114] Optionally, final (x,y) Determined based on the following formula:

[0115]

[0116] in,

[0117] Step 204: The target mask image is mapped to the region image to obtain a target image that only includes the object, and recognition processing is performed on the object based on the target image to obtain a recognition result.

[0118] In an embodiment of the present disclosure, the target mask image is mapped to the region image, the pixels in the region image at the positions corresponding to the objects in the target mask image are set as original pixels, and the pixels at positions other than the objects in the target mask image are set as black pixels, so as to obtain a target image including only the objects.

[0119] For example, take the pedestrian frame image, i.e., the region image, of the pedestrian detection result, correspond the pedestrian mask, i.e., the target mask image, segmented by the SAM model to the pedestrian frame image, then set the pedestrian region pixels to the original pixels and other regions to black RGB (0, 0, 0) pixels to obtain the target image.

[0120] In the embodiment of the present disclosure, after obtaining the target image, the object can be recognized based on the target image to obtain a recognition result.

[0121] In an embodiment of the present disclosure, the electronic device may input a target image into an object re-recognition model to obtain a recognition result, wherein the object re-recognition model may be trained based on a re-recognition dataset.

[0122] In an embodiment of the present disclosure, samples in a re-identification dataset can be obtained based on the following method: determining a training image in an initial re-identification training set, inputting the training image into a multi-scale fusion SAM model, and obtaining a target mask image; obtaining a target image including only the object based on the target mask image, so as to use the target image as a sample in the re-identification dataset, and using the correspondence of the training image as the correspondence of the sample in the re-identification dataset.

[0123] It can be seen that in the embodiment of the present disclosure, the objects to be compared in the object re-identification model are also images that only include the objects. In other words, the objects compared by the re-identification model are all images that only include the objects, which avoids the low recognition accuracy caused by the clutter of background information and improves the accuracy of re-identification.

[0124] In order to better illustrate the technical solutions provided by this disclosure, see Figure 4As shown, the object recognition solution provided by the present disclosure is introduced below with a specific example.

[0125] exist Figure 4 In the embodiment, the electronic device may first perform pedestrian detection processing on the image to be recognized to obtain a pedestrian detection frame image, and then perform SAM pedestrian mask image generation processing based on review and screening on the pedestrian detection frame image (i.e., the process of determining the target mask image in the aforementioned steps 202-203), thereby obtaining a target mask image, and determine the target image based on the target mask image, that is, extracting an image only including pedestrians from the image to be recognized, setting other areas to black or white, and then inputting the target image into the object re-recognition model (e.g., Figure 4 The pedestrian re-identification model in

[15] is used to obtain the similarity sequence.

[0126] It can be seen that in the embodiment of the present disclosure, the pedestrian area, i.e., the aforementioned target image, is segmented with high precision through the SAM pedestrian segmentation method based on multi-scale fusion, so that the input of pedestrian re-identification is not interfered with by the peripheral background in the pedestrian circumscribed frame, thereby improving the accuracy of pedestrian re-identification.

[0127] The exemplary embodiment of the present disclosure also provides an object recognition device. Figure 5 As shown, the object recognition device 500 includes the following program units:

[0128] The receiving unit 501 is configured to determine an image to be recognized;

[0129] an obtaining unit 502 for intercepting a region image of an object in the image to be recognized, and performing scaling processing on the region image to obtain a plurality of scaled images;

[0130] a determining unit 503, configured to determine a target mask image based on the multiple scaled images;

[0131] The recognition unit 504 is configured to match the target mask image to the region image to obtain a target image including only the object, and perform recognition processing on the object based on the target image to obtain a recognition result.

[0132] In a possible implementation, the obtaining unit 502 is configured to:

[0133] Determine multiple preset resolutions;

[0134] The original resolution of the regional image is scaled to the multiple preset resolutions respectively to obtain multiple scaled images.

[0135] In a possible implementation, the determining unit 503 is configured to:

[0136] Inputting the plurality of scaled images into a segmentation model to obtain mask sequences corresponding to the plurality of scaled images, each of the mask sequences including a plurality of mask images and confidence information of each mask image;

[0137] An object mask image in each of the mask sequences is determined, and the target mask image is determined based on the obtained multiple object mask images.

[0138] In a possible implementation, the determining unit 503 is configured to:

[0139] For each mask sequence, do the following:

[0140] Determine pixel aspect ratio information of each mask image in the mask sequence;

[0141] Determining a first candidate set according to pixel aspect ratio information of each mask image and a preset ratio threshold;

[0142] Performing image-text matching processing on each mask image in the first candidate set to determine an object mask image in the mask sequence.

[0143] In a possible implementation, the determining unit 503 is configured to:

[0144] performing scaling processing on the plurality of object mask images to obtain a plurality of adjusted object mask images;

[0145] The target mask image is determined according to the adjusted multiple object mask images.

[0146] In a possible implementation, the determining unit 503 is configured to:

[0147] Creating a blank image of the same size as the region image;

[0148] determining comprehensive pixel position information of the object based on the pixel position information of the object in the adjusted plurality of object mask images and confidence information corresponding to the object mask images;

[0149] Drawing is performed in the blank image according to the integrated pixel position information to determine the target mask image.

[0150] In a possible implementation, the identification unit 504 is configured to:

[0151] The target mask image is mapped to the region image, pixels in the region image corresponding to the position of the object in the mask image are set as original pixels, and pixels in positions other than the object in the mask image are set as black pixels, so as to obtain a target image including only the object.

[0152] The specific details of each part of the above-mentioned device have been described in detail in the implementation method part. The undisclosed details can be found in the implementation method part, so they will not be repeated here.

[0153] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0154] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, implements the object recognition method.

[0155] In one embodiment, a computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The computer-readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk drive (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing the computer program, such as a read-only memory, NAND flash memory, and the like.

[0156] In one embodiment, the computer program product may be an intangible product containing a computer program. For example, the computer program product may be implemented as a virtual digital product, such as a digital file such as an executable file or installation package storing the computer program.

[0157] The code of the computer program can be written in one or more programming languages. Programming languages ​​include C, Java, C++, etc. The program code can be executed entirely on the user computing device, partially on the user computing device, or as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., via an Internet connection provided by a carrier).

[0158] Computer programs can be carried or transmitted through electrical, magnetic, optical, electromagnetic, infrared and other signals. Electronic devices can convert signals carrying computer programs into digital signals, and then run the computer programs. When the computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, it can enable the processor of the electronic device to execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned object recognition method, which includes the following steps: Step 201: Determine the image to be recognized; Step 202: Intercept the regional image of the object in the image to be recognized, and scale the regional image to obtain multiple scaled images; Step 203: Determine the target mask image based on the multiple scaled images; Step 204: Correspond the target mask image to the regional image to obtain a target image that only includes the object, and perform recognition processing on the object based on the target image to obtain a recognition result.

[0159] By implementing the above method steps through a computer program, it is possible to determine the image to be identified, then intercept the regional image of the object in the image to be identified, and perform scaling processing on the regional image to obtain multiple scaled images, thereby determining the target mask image based on the multiple scaled images. Furthermore, the mask image can be mapped to the regional image to obtain a target image that only includes the object, and the object is identified based on the target image to obtain a recognition result. It can be seen that the present disclosure uses a plurality of appropriate scaled images of different scales to achieve accurate segmentation of the object, improve the accuracy of obtaining a target image that only includes the object, and, based on the target image, the object is identified to obtain a recognition result, that is, the influence of background factors on the object re-identification is greatly reduced, thereby improving the recognition accuracy of the object re-identification.

[0160] The exemplary embodiments of the present disclosure further provide an electronic device, which may include a processor and a memory. The memory stores executable instructions for the processor, such as a computer program. The processor executes the executable instructions to perform the method steps of various exemplary embodiments of the present disclosure.

[0161] Reference below Figure 6 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 6 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0162] like Figure 6 As shown, the electronic device 600 may include: a processor 610 , a memory 620 , a bus 630 , an I / O (input / output) interface 640 , and a network adapter 650 .

[0163] The memory 620 may include volatile memory, such as RAM 621 and cache unit 622, and may also include non-volatile memory, such as ROM 623. The memory 620 may also include one or more program modules 624. Such program modules 624 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, the program modules 624 may include the modules in the aforementioned devices.

[0164] The processor 610 may include one or more processing units, for example: the processor 610 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0165] The processor 610 can be used to execute executable instructions stored in the memory 620, such as the above-mentioned object recognition method, which includes the following steps: Step 201: Determine the image to be recognized; Step 202: Intercept the regional image of the object in the image to be recognized, and scale the regional image to obtain multiple scaled images; Step 203: Determine the target mask image based on the multiple scaled images; Step 204: Correspond the target mask image to the regional image to obtain a target image that only includes the object, and perform recognition processing on the object based on the target image to obtain a recognition result.

[0166] By executing the above method steps through the processor 610, the image to be identified can be determined, and then the regional image of the object in the image to be identified can be intercepted, and the regional image can be scaled to obtain multiple scaled images, thereby determining the target mask image based on the multiple scaled images. Furthermore, the mask image can be mapped to the regional image to obtain a target image that only includes the object, and the object can be identified based on the target image to obtain a recognition result. It can be seen that the present disclosure uses a plurality of appropriate scaled images of different scales to achieve accurate segmentation of the object, improve the accuracy of obtaining a target image that only includes the object, and, based on the target image, the object is identified to obtain a recognition result, that is, the influence of background factors on the object re-identification is greatly reduced, thereby improving the recognition accuracy of the object re-identification.

[0167] The bus 630 is used to realize the connection between different components of the electronic device 600 and may include a data bus, an address bus, and a control bus.

[0168] The electronic device 600 can communicate with one or more external devices 700 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 640 .

[0169] The electronic device 600 can communicate with one or more networks via the network adapter 650. For example, the network adapter 650 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. The network adapter 650 can communicate with other modules of the electronic device 600 via the bus 630.

[0170] although Figure 6 Not shown, other hardware and / or software modules may also be provided in the electronic device 600, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0171] As can be seen from the above, the technical solutions of the present disclosure can be implemented as methods, devices, systems, computer program products, storage media, electronic devices, etc. Those skilled in the art will understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, such as "circuit", "module" or "system".

[0172] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are merely exemplary, and the scope and spirit of the present disclosure are indicated by the claims, which should cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the field of the present technology that are not disclosed in the present disclosure.

Claims

1. A method for object recognition, characterized in that: The method comprises: Determine an image to be recognized; intercepting a region image of the object in the image to be recognized, and performing scaling processing on the region image to obtain a plurality of scaled images; determining a target mask image according to the multiple scaled images; The target mask image is mapped to the region image to obtain a target image that only includes the object, and recognition processing is performed on the object based on the target image to obtain a recognition result.

2. The method according to claim 1, characterized in that Scaling the regional image to obtain a plurality of scaled images includes: Determine multiple preset resolutions; The original resolution of the regional image is scaled to the multiple preset resolutions respectively to obtain multiple scaled images.

3. The method according to claim 2, characterized in that Determining a target mask image according to the multiple scaled images includes: Inputting the plurality of scaled images into a segmentation model to obtain mask sequences corresponding to the plurality of scaled images, wherein each mask sequence includes a plurality of mask images and confidence information of each mask image; An object mask image in each of the mask sequences is determined, and the target mask image is determined based on the obtained multiple object mask images.

4. The method according to claim 3, characterized in that Determining an object mask image in each of the mask sequences, comprising: For each mask sequence, do the following: Determine pixel aspect ratio information of each mask image in the mask sequence; Determining a first candidate set according to pixel aspect ratio information of each mask image and a preset ratio threshold; Performing image-text matching processing on each mask image in the first candidate set to determine an object mask image in the mask sequence.

5. The method according to claim 4, characterized in that Determining the target mask image according to the obtained multiple object mask images includes: performing scaling processing on the plurality of object mask images to obtain a plurality of adjusted object mask images; The target mask image is determined according to the adjusted multiple object mask images.

6. The method according to claim 5, characterized in that Determining the target mask image according to the adjusted multiple object mask images includes: Creating a blank image of the same size as the region image; determining comprehensive pixel position information of the object based on the pixel position information of the object in the adjusted plurality of object mask images and confidence information corresponding to the object mask images; Drawing is performed in the blank image according to the integrated pixel position information to determine the target mask image.

7. The method according to any one of claims 1 to 6, characterized in that: Corresponding the target mask image to the region image to obtain a target image including only the object includes: The target mask image is mapped to the region image, pixels in the region image corresponding to the position of the object in the mask image are set as original pixels, and pixels in positions other than the object in the mask image are set as black pixels, so as to obtain a target image including only the object.

8. An object recognition device, characterized in that: The device comprises: A receiving unit, configured to determine an image to be recognized; an obtaining unit, configured to intercept a regional image of an object in the image to be identified, and perform scaling processing on the regional image to obtain a plurality of scaled images; a determining unit, configured to determine a target mask image based on the multiple scaled images; The recognition unit is configured to correspond the target mask image to the region image to obtain a target image including only the object, and perform recognition processing on the object based on the target image to obtain a recognition result.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 7 by executing the executable instructions.

Images