Object recognition method, generation method of reference template for object recognition

Abstract

The application provides an object recognition method and a reference template generation method for object recognition. According to an embodiment of the application, a first image to be recognized is first determined, a reference template group for object recognition of the first image is obtained, and the reference template group is used to perform object recognition on the first image to obtain an object recognition result corresponding to the first image. The reference templates in the reference template group are constructed according to an object recognition result corresponding to a second image associated with the first image. Since at least one reference template in the reference template group records a non-target image region and the non-target region in the reference template is configured with an object recognition confidence, when the object recognition is performed on the first image, the object confidence of the non-target region is used as a reference to reduce the error recognition of the range of the target object region to the non-target region, thereby improving the accuracy of the object recognition result in image recognition and video recognition.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an object recognition method, a method and apparatus for generating reference templates for object recognition, an electronic device, and a storage medium. Background Technology

[0002] Object recognition in videos is a common task in computer vision. The goal is to identify the position and size of a specified object in a given frame of a video, and then to determine the position and size of that same object in other video frames. Video object recognition has wide applications in real life, such as smart cities, traffic management, and drone tracking, bringing many conveniences to people's lives.

[0003] Currently, the accuracy of object recognition needs to be improved for more complex application scenarios. Furthermore, as time changes in the video, the appearance features of the target object to be recognized, such as shape, size, color, and posture, may change. These changes can affect the accuracy of object recognition and may even lead to object recognition failure. Summary of the Invention

[0004] This application provides an object recognition method and a method for generating reference templates for object recognition. Based on the reference templates in the reference template group, the target object in the first image is identified to improve the accuracy of object recognition and avoid object recognition failure due to changes in the appearance of the target object.

[0005] In a first aspect, embodiments of this application provide an object identification method, the method comprising:

[0006] Determine the first image to be identified;

[0007] A reference template group for object recognition of the first image is obtained; the reference templates in the reference template group are constructed based on the object recognition results of the second image associated with the first image, wherein at least one reference template records a non-target image region where no target object exists, and the non-target image region is configured with a corresponding object recognition confidence level;

[0008] The obtained reference template group is used to perform object recognition on the first image to obtain the object recognition result corresponding to the first image.

[0009] Secondly, embodiments of this application provide an object recognition method, the method comprising:

[0010] Display at least one image frame in the video and obtain the object coordinate range based on the at least one image frame;

[0011] The object recognition results of other image frames in the video are obtained according to the object coordinate range; the object recognition results of other image frames are obtained based on the reference template group, the reference templates in the reference template group are constructed according to the object recognition results of the image frames associated with the other image frames, wherein at least one reference template records non-target image regions where no target object exists, the non-target image regions are configured with corresponding object recognition confidence, and the non-target image regions in the reference templates of at least one other image frame are determined according to the object coordinate range;

[0012] Other image frames labeled with the object recognition results are displayed.

[0013] Thirdly, embodiments of this application provide a method for generating a reference template for object recognition, the method comprising:

[0014] Obtain at least one object recognition result corresponding to the second image;

[0015] A reference template group is constructed based on the obtained object recognition results. At least one reference template in the reference template group records a non-target image region where no target object exists. The non-target image region is configured with a corresponding object recognition confidence level. The reference template group is used for object recognition of a first image associated with the second image to obtain an object recognition result corresponding to the first image.

[0016] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method described in any of the above-mentioned embodiments.

[0017] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0018] Compared with the prior art, this application has the following advantages:

[0019] According to the embodiments of this application, a first image to be identified is first determined, a reference template group for object recognition of the first image is obtained, and the reference template group is used to perform object recognition on the first image to obtain an object recognition result corresponding to the first image. The reference templates in the reference template group are constructed based on the object recognition result corresponding to a second image associated with the first image. Since at least one reference template in the reference template group records a non-target image region and the non-target region in the reference template is configured with an object recognition confidence level, when performing object recognition on the first image, the object confidence level of the non-target region can be used as a reference to reduce erroneous identification of the target object's region as a non-target region, thereby improving the accuracy of object recognition results in image recognition and video recognition.

[0020] Furthermore, the reference template can also record the target image region containing the target object and the corresponding object recognition confidence level. Multiple reference templates constructed from the object recognition results corresponding to multiple second images can be added to the reference template group. Taking video recognition as an example, since different second images come from image frames at different time points in the video, the appearance features (shape, size, color, posture, etc.) of the target object will also change at different time points in the video. Therefore, multiple reference templates constructed corresponding to the second images can represent the appearance features of the target object at different time points in the video. Thus, the target object whose appearance has changed can be identified from the first image based on the reference template group, improving the accuracy of object recognition in the corresponding video.

[0021] While obtaining the object recognition result of the first image, a new reference template based on the corresponding object recognition result of the first image can be directly constructed and added to the reference template group. This allows for timely updates to the reference template group, enabling it to represent the latest appearance features of the target object and thus providing more accurate object recognition results for subsequent recognition. In online object recognition scenarios, updating the reference template group by deleting previously added reference templates after adding them can control the number of computations during object recognition while maintaining accuracy, thereby improving the efficiency of object recognition.

[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0023] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0024] Figure 1 This is a schematic diagram of a scenario for the object recognition scheme provided in this application;

[0025] Figure 2 This is a flowchart of an object recognition method according to an embodiment of this application;

[0026] Figure 3 This is a flowchart of an object recognition method according to another embodiment of this application;

[0027] Figure 4 This is a flowchart of a method for generating a reference template for object recognition according to an embodiment of this application;

[0028] Figure 5 This is a structural block diagram of an object recognition device according to an embodiment of this application;

[0029] Figure 6 This is a structural block diagram of an object recognition device according to another embodiment of this application;

[0030] Figure 7 This is a structural block diagram of a reference template generation apparatus for object recognition according to an embodiment of this application; and

[0031] Figure 8 This is a block diagram of an electronic device used to implement the embodiments of this application. Detailed Implementation

[0032] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0033] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0034] In a related technology prior to this application, object recognition of a target object in a first image was performed using only a single reference template. Since a single reference template only records the appearance features of the target object at a specific point in time within the video, it cannot adequately describe the changes in the target object's appearance, thus affecting the accuracy of the object recognition results.

[0035] In view of this, embodiments of this application provide a new object recognition scheme to solve all or part of the above-mentioned technical problems.

[0036] The object recognition scheme provided in this application is used to identify target objects in videos or images. This scheme can be applied to fields such as security (e.g., vehicle tracking scenarios), video monitoring (e.g., motion trajectory recognition scenarios, pet monitoring scenarios), and inspection (e.g., drone tracking scenarios, robot navigation scenarios). Specifically, in vehicle tracking scenarios, a vehicle can be identified in a monitoring video captured by cameras deployed at road monitoring points; in pet monitoring scenarios, pets can be identified in monitoring videos captured by home monitoring devices; in drone tracking scenarios, the target object (such as a vehicle, person, or animal) that the drone is tracking in a video captured by the drone can be identified, and the identified target object can be tracked.

[0037] Figure 1 This is a schematic diagram illustrating an application scenario for implementing the method of the embodiments of this application. Figure 1 This diagram illustrates real-time object recognition using embodiments of this application in a monitoring scenario employing a PTZ camera. Figure 1 As shown, the latest frame captured by the PTZ camera is designated as the first image. The first image and multiple reference templates from a reference template group are input into a deep learning model (Transformer) employing a self-attention mechanism. The input reference templates can include templates with only target regions, templates with only non-target regions, and templates with both target and non-target regions. Each target and non-target region is configured with a corresponding object recognition confidence level. The self-attention layer in the Transformer assigns weights to each reference template in the reference template group; reference templates with higher weights have a greater impact on object recognition in the first image. The Transformer can also extract image features from the first image and multiple reference templates in the reference template group, and calculate the template similarity between the first image and the reference template group based on these features. The coordinate point with the highest template similarity value in the first image is determined as the center point of the bounding box of the target object in the first image, thereby determining the object recognition result of the first image. After obtaining the object recognition results of the first image, the target object can be marked on the first image based on the object recognition results. For example, the bounding box of the target object can be marked on the first image, and the marked first image can be provided to the management of the PTZ camera equipment or the client equipment of the viewer of the monitoring video. New reference templates can also be constructed based on the object recognition results of the first image, and these new reference templates can be added to a reference template group.

[0038] It is understandable that the manager of the PTZ camera equipment or the viewer of the monitoring video can delineate the target object on the image frame (e.g., the first frame of the video) in the video on the client device associated with the PTZ camera equipment. Before performing object recognition on the video, the image frame with the delineation result can be obtained, at least two initial reference templates can be constructed based on the delineation result, and the constructed reference templates can be added to the reference template group.

[0039] The object recognition scheme provided in this application can be applied to image recognition and video recognition. Video recognition scenarios can further include online and offline recognition. Online recognition refers to identifying target objects in a first image based on image frames preceding the first image in the video. Offline recognition refers to identifying target objects in a first image based on image frames excluding those preceding and / or following the first image in the video. For online recognition scenarios, this scheme can also be further applied to real-time recognition, that is, real-time object recognition of monitored or captured videos.

[0040] This application provides an object recognition method, such as... Figure 2 The diagram shown is a flowchart of an object recognition method 200 according to an embodiment of this application. The method 200 may include:

[0041] In step S201, the first image to be identified is determined.

[0042] In image recognition, the image to be recognized is designated as the first image. In video recognition, the image frame in the video for which object recognition is to be performed is designated as the first image. In online recognition scenarios, object recognition can be performed on the image frames in the video sequentially according to the video's chronological order. In this case, the first image refers to the earliest frame in the video that has not yet undergone object recognition. For real-time recognition, the newest image frame received in real-time can be designated as the first image. In offline recognition scenarios, the first image can be any image frame in the video. When determining the first image, it can be either the earliest frame in the video that has not yet undergone object recognition, or a frame selected by the client from the video can be chosen as the first image.

[0043] In step S202, a reference template group for object recognition of the first image is obtained; the reference templates in the reference template group are constructed based on the object recognition results of the second image associated with the first image, wherein at least one reference template records a non-target image region where no target object exists, and the non-target image region is configured with a corresponding object recognition confidence level.

[0044] After determining the first image, a reference template set for object recognition of the first image is obtained. The reference template set includes multiple reference templates used for object recognition in the image. These templates are constructed based on a second image, which refers to one or more other images associated with the first image. This association can mean belonging to the same video or having content association (corresponding to the same object or the same background). For example, the second image might be an image in the same video that precedes the first image in a time series, or a specified image that has content association with the first image. After object recognition is completed on the second image, the object recognition result is obtained, indicating the position (e.g., coordinate range) of the target object in the image. It is understandable that, taking video recognition as an example, if the target object changes dynamically in the video, its position in each frame may be different, and it may even be absent from some images.

[0045] This application's embodiments distinguish different regions within an image (or an image frame corresponding to a video recognition scenario). Based on the object recognition result, the region where the target object is located is designated as the target image region, and the region where no target object exists is designated as a non-target image region. Corresponding object recognition confidence scores are configured for the target image region and the non-target image region. Here, the object recognition confidence score refers to the probability value that a target object can be identified in a certain region of the image. Therefore, when performing object recognition on the first image using a reference template group, using the object confidence score of the non-target region as a reference reduces the erroneous identification of the target object's region as a non-target region, thus improving the accuracy of the object recognition result.

[0046] The following will explain in detail the process of generating reference templates in the reference template group.

[0047] In one implementation, when generating the reference template in the reference template group, the object recognition result corresponding to the second image is first obtained. The following is an example of a possible object recognition result: [pb x b y b h b w The value 'p' indicates whether a target object exists in the second image. When p = 0, it means that the target object does not exist in the second image, and the value 'b' in the object recognition result is not considered. x b y b h and b w All are meaningless terms. When p = 1, it indicates that the target object exists in the second image. x b y bh and b w This is used to indicate the coordinate range of the target object in the second image. This coordinate range can be the bounding rectangle of the target object, that is, the coordinate range of the target object's bounding box. Where, b x and b y These are the coordinates of the center point of the bounding box of the target object, and are b. h and b w These are the height and width of the bounding box of the target object, respectively.

[0048] After obtaining the object recognition result corresponding to the second image, at least a portion of the image region outside the target image region indicated by the object recognition result is defined as a non-target image region. If the object recognition result indicates that there is no target object in the second image, the entire image region of the second image can be defined as a non-target region. If the object recognition result indicates that there is a target object in the second image, at least a portion of the image region outside the target range indicated by the object recognition result can be defined as a non-target image region based on the target object's coordinate range. It is understood that other objects similar to the target object may exist in the second image, and these other objects also have corresponding coordinate ranges. In this case, the coordinate ranges corresponding to these other objects can be defined as non-target image regions.

[0049] After identifying the non-target image region in the second image, the object recognition confidence corresponding to the non-target image region is determined from the object recognition results, and a first reference template is generated to record the non-target image region and the object recognition confidence corresponding to the non-target image region.

[0050] In one embodiment, the first reference template may also record the target image region and the corresponding object recognition confidence level. That is, the first reference template may simultaneously record the target image region and non-target image regions, as well as the object recognition confidence levels corresponding to each of these two types of regions. Correspondingly, the generation process of the reference template in the aforementioned reference template group also includes determining the object recognition confidence level corresponding to the target image region. The target image region is determined by the coordinate range of the target object in the second image; it is understood that its corresponding object recognition confidence level is higher than that of the non-target region. Therefore, when using the first reference template to perform object recognition on the first image, image regions in the first image that have a high similarity to the target image region in the first reference template or a low similarity to the non-target image region can be considered image regions containing the target object, while image regions that have a low similarity to the target image region in the first reference template or a high similarity to the non-target image region can be considered image regions where the target object does not exist.

[0051] In one embodiment, a second reference template can be generated that records only the target image region and its corresponding confidence level. This template is used to perform object recognition in conjunction with the first reference template. In the first image, image regions with a high similarity to the target image region in the second reference template can be considered as image regions containing the target object, while image regions with a low similarity to the target image region can be considered as image regions where the target object does not exist. Accordingly, the process of generating the reference template in the above-mentioned reference template group can further include determining the target image region and the object recognition confidence level corresponding to the target image region from the object recognition result. A second reference template is generated that records the target image region and the object recognition confidence level corresponding to the target image region. That is, unlike the first reference template, the second reference template only includes the target image region containing the target object and its corresponding object recognition confidence level.

[0052] In one implementation, when acquiring a reference template set for object recognition of a first image, one or more image frames in the same video preceding the first image can be identified as associated second images. For online recognition scenarios, since the acquisition of the second image is time-limited, the second image used to construct the reference template can be an image frame in the video that precedes the first image in a time sequence. It is understood that in offline recognition scenarios, since the acquisition of the second image is not time-limited, the identified second image can be one or more image frames that precede and / or follow the first image.

[0053] After determining the second image, a reference template is obtained based on the object recognition result corresponding to the second image, and the constructed reference template is added to the reference template group. Since different second images are image frames from different time points in the video, the appearance features (shape, size, color, posture, etc.) of the target object will also change at different time points in the video. Therefore, the multiple reference templates constructed corresponding to the second image can represent the appearance features of the target object at different time points in the video. Thus, the target object whose appearance has changed can be identified from the first image based on the reference template group, thereby improving the accuracy of object recognition in the corresponding video.

[0054] In one embodiment, a new reference template can be constructed based on the object recognition result corresponding to the first image, and the new reference template can be added to the reference template group. That is, after completing the object recognition of the first image, one or more new reference templates can be constructed based on the object recognition result of the first image. The new reference template is a reference template constructed from the object recognition result of the first image. The new reference template can be the first reference template that records the target image region and / or non-target image region and the corresponding object recognition confidence, or it can be a second reference template that only records the target image region and the object recognition confidence corresponding to that region, or both types of new reference templates can be added simultaneously.

[0055] In one implementation, reference templates added earlier in the reference template group can also be deleted. That is, the reference templates in the reference template group can be updated, and reference templates added earlier can be deleted. On the one hand, by deleting reference templates added earlier in the reference template group, the number of reference templates in the reference template group can be controlled, thereby controlling the number of operations when implementing the object recognition method, and thus improving the efficiency of object recognition. On the other hand, for object recognition in online recognition scenarios, since the position, size, and appearance features of the target object change over time, that is, the changes in the characteristics of the target object are time-dependent, when performing object recognition based on reference templates added later in the reference template group compared to reference templates added earlier, the second image used to construct the reference template is closer to the current first image in the time series, and therefore more relevant, thus allowing the first image to obtain a more accurate recognition result. When deleting an earlier template from a reference template group, you can pre-set the number of reference templates in the group, for example, setting the number to 200, and deleting the first added reference template when the 201st reference template is added; or you can set the deletion quantity, for example, setting the deletion quantity to 1, and deleting the earliest added reference template in the group when adding one or more newly built reference templates.

[0056] In one embodiment, the target object can be marked in the first image based on the object recognition result corresponding to the first image, so that the target object can be seen intuitively when the first image is displayed subsequently. Specifically, the target object can be marked by drawing a bounding box in the first image based on the coordinate range of the target object indicated by the object recognition result. In addition, the motion trajectory of the target object can be generated based on the object recognition results corresponding to multiple image frames (or a group of images with a temporal relationship) in the same video. For example, the center point coordinates of the bounding box of the target object can be obtained first through the object recognition results corresponding to multiple image frames, and then the motion trajectory of the target object can be generated based on the center point coordinates of the bounding box of the target object corresponding to multiple image frames, combined with the time series corresponding to the above multiple image frames.

[0057] Here are some possible application scenarios. In drone tracking, the drone's flight path can be adjusted based on the target object's movement trajectory generated by object recognition, ensuring the drone consistently tracks the target. In pet monitoring, the camera's direction can be adjusted based on the target object's movement trajectory generated by object recognition, ensuring the camera follows the pet's movement and keeps the pet near the center of the monitored area. In intelligent video monitoring, target objects can be marked with bounding boxes in the monitoring video, allowing client users to monitor and track them. Furthermore, the generated movement trajectories can be used for motion trajectory analysis, enabling risk warnings based on abnormal movements.

[0058] In step S203, the obtained reference template group is used to perform object recognition on the first image to obtain the object recognition result corresponding to the first image.

[0059] In one implementation, when performing object recognition on a first image using the acquired reference template set to obtain the object recognition result corresponding to the first image, the template similarity between each reference template in the acquired reference template set and the first image can be determined first. Template similarity refers to the similarity between pixels corresponding to coordinate points between the reference template and the first image, and can be used to indicate the object recognition result of the target object in the first image. For example, if the determined template similarity between each reference template and the first image is low, it can be considered that there is no target object in the first image. When representing image frames in a video in vector form, the template similarity can be determined by calculating the similarity between the vector of the first image and the vector of the reference template.

[0060] Then, a template group similarity score is generated between the reference template group and the first image based on the template similarity and the weights of the reference templates. The weights of the reference templates are obtained by assigning them using the self-attention layer of the deep learning model. That is, a deep learning model employing a self-attention mechanism can assign weights to the reference templates in the reference template group. The self-attention mechanism can learn the correlation between the reference templates in the reference template group and assign greater weights to reference templates that provide stronger references, thereby improving the accuracy of object recognition in the first image. In one possible implementation, the coordinate point with the highest template group similarity score in the first image can be determined as the center point of the bounding box of the target object in the first image, thus determining the object recognition result of the first image.

[0061] In one embodiment, the reference template further records a target image region and an object recognition confidence level corresponding to the target image region. When determining the template similarity between each reference template in the acquired reference template group and the first image, the first template similarity between the first image and the reference template within the defined coordinate range can be determined firstly, according to the coordinate range defined by the target image region recorded in the reference template. When the first template similarity is sufficient to determine the target object, the object recognition result of the first image can be directly determined based on the first template similarity. When the target object cannot be determined solely based on the first template similarity, the second template similarity between the target first image and the reference template within the defined coordinate range is determined according to the coordinate range defined by the non-target image region recorded in the reference template. Then, the template similarity between each reference template and the first image is determined based on the first template similarity and the second template similarity, and the object recognition result corresponding to the first image is obtained based on the template similarity. Thus, when determining the position of the target object in the first image, the more referential target image region can be used as the primary basis, thereby improving the accuracy of object recognition.

[0062] This application also provides another object recognition method. Figure 3 This is a flowchart of an object recognition method 300 according to an embodiment of this application. The method 300 may include:

[0063] In step S301, at least one image frame in the video is displayed, and the range of object coordinates input based on the at least one image frame is obtained.

[0064] In one possible implementation, the bounding box that the user has drawn around the target object on the displayed image frame can be obtained first, and then the coordinate range of the target object in the image frame can be obtained by obtaining the coordinate range of the bounding box.

[0065] In step S302, the object recognition results of other image frames in the video are obtained according to the object coordinate range. The object recognition results of other image frames are obtained based on a reference template group. The reference templates in the reference template group are constructed based on the object recognition results of the image frames associated with the other image frames. At least one reference template records non-target image regions where no target object exists. The non-target image regions are configured with corresponding object recognition confidence. The non-target image regions in the reference templates of at least one other image frame are determined based on the object coordinate range.

[0066] Other image frames refer to image frames in the video for which object recognition results have not been obtained, such as the latest image frame captured by the video capture device in real-time recognition. Specific methods for obtaining object recognition results for other image frames can be found in the embodiments provided by method 200, and will not be repeated here.

[0067] In step S303, other image frames marked with the object recognition results are displayed.

[0068] After obtaining object recognition results from other image frames in the video, these other image frames, labeled with the object recognition results, can be displayed on client devices (such as desktop computers, tablets, and mobile phones). Furthermore, in the case of real-time recognition, the video labeled with the object recognition results can be displayed on the client in real time.

[0069] This application also provides a method for generating a reference template for object recognition, such as... Figure 4 The diagram shows a flowchart of a method 400 for generating a reference template for object recognition according to an embodiment of this application. Method 400 may include:

[0070] In step S401, at least one object recognition result corresponding to the second image is obtained.

[0071] In step S402, a reference template group is constructed based on the obtained object recognition results. At least one reference template in the reference template group records a non-target image region where no target object exists. The non-target image region is configured with a corresponding object recognition confidence level. The reference template group is used for object recognition of a first image associated with the second image to obtain an object recognition result corresponding to the first image.

[0072] The method for generating reference templates for object recognition provided in this application corresponds to the process of generating reference templates in the reference template group provided in method 200 above, and will not be described again here.

[0073] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide an object recognition device. For example... Figure 5 This is a structural block diagram of an object recognition device 500 according to an embodiment of this application. The object recognition device 500 may include:

[0074] Image determination module 501 is used to determine the first image to be identified;

[0075] Template group acquisition module 502 acquires a reference template group for object recognition of the first image; the reference templates in the reference template group are constructed based on the object recognition results of the second image associated with the first image, wherein at least one reference template records a non-target image region where no target object exists, and the non-target image region is configured with a corresponding object recognition confidence level;

[0076] The first result acquisition module 503 is used to perform object recognition on the first image using the acquired reference template group to obtain the object recognition result corresponding to the first image.

[0077] In one embodiment, the apparatus 500 may further include a reference template generation module, the reference template generation module comprising:

[0078] The second result acquisition submodule is used to acquire the object recognition result corresponding to the second image;

[0079] The region determination submodule is used to determine at least a portion of the image region outside the target image region where the target object is indicated by the object recognition result as a non-target image region;

[0080] The first confidence determination submodule is used to determine the object recognition confidence corresponding to the non-target image region from the object recognition result;

[0081] The reference template generation submodule is used to generate a first reference template that records the non-target image region and the object recognition confidence corresponding to the non-target image region.

[0082] In one embodiment, the reference template generation module further includes:

[0083] The second confidence determination submodule is used to determine the object recognition confidence corresponding to the target image region. The first reference template also records the target image region and the object recognition confidence corresponding to the target image region.

[0084] In one embodiment, the reference template generation module further includes:

[0085] The third confidence determination submodule determines the target image region and the object recognition confidence corresponding to the target image region from the object recognition result; and generates a second reference template that records the target image region and the object recognition confidence corresponding to the target image region.

[0086] In one embodiment, the template group acquisition module 502 may include:

[0087] The second image determination submodule determines one or more image frames in the same video that precede the first image as the associated second image;

[0088] The template addition submodule is used to obtain a reference template constructed based on the object recognition result corresponding to the second image and add it to the reference template group.

[0089] In one embodiment, the device 500 may further include:

[0090] The reference template construction module is used to construct a new reference template based on the object recognition result corresponding to the first image, and add the new reference template to the reference template group.

[0091] In one implementation, the reference template construction module can be specifically used to delete reference templates that were added to the reference template group earlier in the time they were added.

[0092] In one embodiment, the device 500 may further include:

[0093] The target object marking module is used to mark the target object in the first image according to the object recognition result corresponding to the first image; and / or, the motion root trajectory generation module is used to generate the motion trajectory of the target object according to the object recognition results corresponding to multiple image frames in the same video.

[0094] In one embodiment, the step of using the acquired reference template set to perform object recognition on the first image to obtain an object recognition result corresponding to the first image includes:

[0095] The template similarity determination submodule is used to determine the template similarity between each reference template in the acquired reference template group and the first image;

[0096] The template group similarity determination submodule is used to generate a template group similarity between the reference template group and the first image based on the template similarity and the weight of the reference template, wherein the weight of the reference template is obtained by calling the self-attention layer of the deep learning model.

[0097] In one embodiment, the reference template further records the target image region and the object recognition confidence level corresponding to the target image region;

[0098] The template similarity determination submodule can be specifically used to: determine the first template similarity between the target first image and the reference template within the defined coordinate range of the target image region recorded in the reference template; determine the second template similarity between the target first image and the reference template within the defined coordinate range of the non-target image region recorded in the reference template; and determine the template similarity between each reference template and the first image based on the first template similarity and the second template similarity.

[0099] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide another object recognition device. For example... Figure 6 This is a structural block diagram of an object recognition device 600 according to an embodiment of this application. The object recognition device 600 may include:

[0100] The coordinate range input module 601 is used to display at least one image frame in the video and obtain the object coordinate range based on the at least one image frame.

[0101] The result acquisition module 602 is used to acquire object recognition results of other image frames in the video based on the object coordinate range; the object recognition results of other image frames are obtained based on a reference template group, the reference templates in the reference template group are constructed based on the object recognition results of the image frames associated with the other image frames, wherein at least one reference template records non-target image regions where no target object exists, the non-target image regions are configured with corresponding object recognition confidence, and the non-target image regions in the reference templates of at least one other image frame are determined based on the object coordinate range;

[0102] The image frame display module 603 is used to display other image frames marked with the object recognition results.

[0103] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide an apparatus for generating reference templates for object recognition. For example... Figure 7 This is a structural block diagram of a reference template generation apparatus 700 for object recognition according to an embodiment of this application. The reference template generation apparatus 700 for object recognition may include:

[0104] Result acquisition module 701 is used to acquire at least one object recognition result corresponding to a second image;

[0105] The template group construction module 702 is used to construct a reference template group based on the obtained object recognition results. At least one reference template in the reference template group records a non-target image region where no target object exists. The non-target image region is configured with a corresponding object recognition confidence level. The reference template group is used for object recognition of a first image associated with the second image to obtain an object recognition result corresponding to the first image.

[0106] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0107] Figure 8 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 8 As shown, the electronic device includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the computer program, it implements the method described in the above embodiments. The number of memories 801 and processors 802 can be one or more.

[0108] The electronic device also includes:

[0109] The communication interface 803 is used to communicate with external devices and exchange and transmit data.

[0110] If the memory 801, processor 802, and communication interface 803 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0111] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0112] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0113] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0114] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0115] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0116] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0117] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0118] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0120] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0121] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0122] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0124] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An object recognition method, characterized in that, The method includes: Determine the first image to be identified; A reference template set for object recognition of a first image is obtained; the reference template set includes at least a first reference template, the first reference template recording non-target image regions in a second image and object recognition confidence corresponding to the non-target image regions, the non-target image regions including at least a portion of the image regions outside the target image regions where target objects exist as indicated by the object recognition result corresponding to the second image, and the object recognition confidence corresponding to the non-target image regions is determined from the object recognition result; The obtained reference template group is used to perform object recognition on the first image to obtain the object recognition result corresponding to the first image.

2. The method according to claim 1, characterized in that, The process of generating the reference template in the reference template group also includes: The object recognition confidence level corresponding to the target image region is determined, and the first reference template also records the target image region and the object recognition confidence level corresponding to the target image region.

3. The method according to claim 1, characterized in that, The process of generating the reference template in the reference template group also includes: The target image region and the object recognition confidence level corresponding to the target image region are determined from the object recognition results. A second reference template is generated to record the target image region and the object recognition confidence corresponding to the target image region.

4. The method according to claim 1, characterized in that, The step of obtaining the reference template set for object recognition of the first image includes: One or more image frames in the same video that precede the first image are identified as the associated second image; Obtain the reference template constructed based on the object recognition result corresponding to the second image, and add it to the reference template group.

5. The method according to claim 1, characterized in that, The method further includes: A new reference template is constructed based on the object recognition result corresponding to the first image, and the new reference template is added to the reference template group.

6. The method according to claim 5, characterized in that, The method further includes: Delete the reference template that was added to the reference template group earlier than the one that was added earlier.

7. The method according to claim 1, characterized in that, The method further includes: Mark the target object in the first image based on the object recognition result corresponding to the first image; And / or, generate the motion trajectory of the target object based on the object recognition results corresponding to multiple image frames in the same video.

8. The method according to claim 1, characterized in that, The step of using the acquired reference template set to perform object recognition on the first image to obtain the object recognition result corresponding to the first image includes: Determine the template similarity between each reference template in the acquired reference template group and the first image; The template similarity between the reference template group and the first image is generated based on the template similarity and the weight of the reference template, wherein the weight of the reference template is obtained by calling the self-attention layer of the deep learning model.

9. The method according to claim 8, characterized in that, The reference template also records the target image region and the object recognition confidence level corresponding to the target image region; The determination of the template similarity between each reference template in the acquired reference template group and the first image includes: Based on the coordinate range defined by the target image region recorded in the reference template, determine the first template similarity between the first image and the reference template within the defined coordinate range; Based on the coordinate range defined by the non-target image region recorded in the reference template, determine the second template similarity between the first image and the reference template within the defined coordinate range; The template similarity between each reference template and the first image is determined based on the first template similarity and the second template similarity.

10. An object recognition method, characterized in that, The method includes: Display at least one image frame in the video and obtain the object coordinate range based on the at least one image frame; The object recognition results of other image frames in the video are obtained based on the object coordinate range; the object recognition results of other image frames are obtained based on a reference template group, the reference template group includes at least a first reference template, the first reference template records non-target image regions in the image frames associated with the other image frames and the object recognition confidence corresponding to the non-target image regions, the non-target image regions include at least a portion of the image regions outside the target image regions indicated by the object recognition results of the associated image frames, and the object recognition confidence corresponding to the non-target image regions is determined from the object recognition results; Other image frames labeled with the object recognition results are displayed.

11. A method for generating a reference template for object recognition, characterized in that, The method includes: Obtain at least one object recognition result corresponding to the second image; A reference template group is constructed based on the obtained object recognition results. The reference template group includes at least a first reference template. The first reference template records non-target image regions in the second image and object recognition confidence corresponding to the non-target image regions. The non-target image regions include at least a portion of the image regions other than the target image regions where the target object is indicated by the object recognition results corresponding to the second image. The object recognition confidence corresponding to the non-target image regions is determined from the object recognition results.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-11.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-11.

Images