An object re-identification method, device, electronic equipment and system

Abstract

Embodiments of the present application provide an object re-identification method and device, electronic equipment and system, and relate to the technical field of machine vision. The object re-identification method comprises: acquiring a to-be-identified video frame in a first video to be re-identified; determining a current to-be-matched region from an object image region in the to-be-identified video frame; if the current to-be-matched region does not match a template image, adding the current to-be-matched region to a first candidate region set and continuing to determine the current to-be-matched region; if the current to-be-matched region matches the template image, adding the current to-be-matched region to a first matching region set of a specified object, adding a first candidate region in the first candidate region set that matches the current to-be-matched region to the first matching region set, and continuing to determine the current to-be-matched region. In this way, the probability of missed detection can be reduced, and a re-identification result with high accuracy can be obtained.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to an object re-identification method, apparatus, electronic device and system. Background Technology

[0002] With the development of machine vision technology, it is possible to track objects across cameras. For example, based on REID (Person Re-Identification) technology, image regions containing people can be detected in video frames captured by various cameras in a scene. Image regions belonging to the same person can be associated to obtain re-identification results. Subsequently, the movement path of the same person can be determined based on the re-identification results.

[0003] It is evident that accurately identifying the image region of the same object to obtain highly accurate re-identification results has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this application is to provide an object re-identification method, apparatus, electronic device, and system to obtain re-identification results with high accuracy. The specific technical solution is as follows:

[0005] In a first aspect of this application, an object re-identification method is provided. The method includes: acquiring a video frame to be re-identified from a first video, as the video frame to be identified; determining, from the object image region occupied by the object in the video frame to be identified, the next object image region to be matched with a template image of a specified object, as the current region to be matched; if the current region to be matched does not match the template image, adding the current region to a first candidate region set, and returning to the step of determining, from the object image region occupied by the object in the video frame to be identified, the next object image region to be matched with a template image of a specified object; if the current region to be matched matches the template image, adding the current region to a first matching region set of the specified object, and adding the first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set, and returning to the step of determining, from the object image region occupied by the object in the video frame to be identified, the next object image region to be matched with a template image of a specified object.

[0006] In a second aspect of this application, an object re-identification apparatus is provided, the apparatus comprising:

[0007] The acquisition module is used to acquire the video frames to be re-identified in the first video, as the video frames to be identified.

[0008] The region determination module is used to determine the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified, and use it as the current region to be matched.

[0009] The first matching module is used to add the current region to be matched to the first candidate region set if the current region to be matched does not match the template image, and return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0010] The second matching module is used to add the current region to be matched to the first matching region set of the specified object if the current region to be matched matches the template image, and add the first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set, and return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0011] In a third aspect of this application, an object re-identification system is provided. The system includes: at least one first acquisition device for acquiring a first video frame containing an object image region; at least one second acquisition device for acquiring a second video frame containing an object image region; and a processing device for receiving the first video frame acquired by the first acquisition device and the second video frame acquired by the second acquisition device, respectively, and implementing the object re-identification method described in any of the first aspects above.

[0012] In a fourth aspect of the embodiments of this application, an electronic device is provided, comprising: a memory for storing a computer program; and a processor for executing the program stored in the memory to implement the object re-identification method described in any of the first aspects above.

[0013] In a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the object re-identification method described in any of the first aspects above.

[0014] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the object re-identification method described in any of the first aspects above.

[0015] Based on the object re-identification method provided in this application embodiment, the video frames to be re-identified in the first video include object image regions occupied by objects. For the object image region (i.e., the current matching region) for which it needs to be determined whether it belongs to a specified object, the electronic device first matches the object image region with the template image of the specified object. If the object image region does not match the template image, it is added to the first candidate region set, i.e., the object image region is first used as the first candidate region. If the object image region matches the template image, it is added to the first matching region set, i.e., the object image region is used as the first matching region of the specified object, and then matched with the first candidate region set. For the first candidate region that does not match the template image, the first matching region that matches the template image can be used again to match the first candidate region, thereby re-filtering the first candidate region and determining whether it belongs to the specified object, i.e., identifying the missed first matching regions. The first matching region in the final first matching region set is the object image region belonging to the specified object determined by the electronic device for object re-identification of the first video. In this way, the probability of missed detection can be reduced, and more accurate re-identification results can be obtained.

[0016] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0018] Figure 1 This is a first flowchart of an object re-identification method provided in an embodiment of this application;

[0019] Figure 2 A flowchart illustrating the object re-identification result within a single segment obtained based on a candidate pool mechanism, as provided in this application embodiment;

[0020] Figure 3 A flowchart illustrating the object re-identification mechanism with inter-fragment sample error correction provided in this application embodiment;

[0021] Figure 4 A structural diagram of the object re-identification device provided in the embodiments of this application;

[0022] Figure 5 An architecture diagram of an object re-identification system provided in this application embodiment;

[0023] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0025] With the development of machine vision technology, it is possible to track objects across cameras (a process known as object re-identification). The object can be a person, a vehicle, or an animal. Taking a person as an example, REID technology can be used to re-identify the person based on video frames captured by various cameras in the scene, and the movement path of the same person can be determined based on the re-identification results.

[0026] In real-world scenarios, the movement routes of the same person identified using REID technology can be further processed based on business needs. For example, in an indoor scenario with an indoor camera, the movement routes of the same person indoors can be analyzed to understand their habits and provide personalized smart home services. Alternatively, in an outdoor scenario with an outdoor camera and a missing person, the movement routes of the missing person outdoors can be used to assist in locating them.

[0027] To improve the accuracy of the re-identification results, when performing object re-identification, information such as route information and object pose information from the video can be integrated to jointly determine the image regions belonging to the same object. In model-based object re-identification, improving the accuracy of the results can be considered as enhancing the performance of the algorithm model.

[0028] In real-world scenarios, after obtaining the re-identification results, further post-processing can be performed by optimizing feature matching, data filtering, result verification, and improving ranking algorithms to overcome interference in the scene. Such interference may include differences in camera perspectives, variations in lighting within the field of view of different cameras, and occlusion within the scene. This can further improve the accuracy of identifying image regions belonging to the same object and enhance the stability of object re-identification. Consequently, it can improve the stability of scenarios where object re-identification technology is applied (such as using public safety cameras for public safety and traffic management).

[0029] Furthermore, the aforementioned optimized feature matching can extract more stable local features using deep learning models. For example, when the object is a person, the local features of the person can represent clothing texture, backpack, etc. The aforementioned ranking algorithm improvement can be used to combine multimodal information for post-processing, such as image information and temporal information of video sequences. In the post-processing stage, by optimizing feature matching and improving the ranking algorithm, mismatches caused by factors such as changes in lighting and differences in person pose can be reduced during the aforementioned object re-identification process, thus identifying more accurate image regions belonging to the same object.

[0030] In real-world scenarios, models can be used for the aforementioned object re-identification and post-processing. These models can be called re-identification models, including the aforementioned models for object re-identification and the aforementioned deep learning models. However, real-world scenarios may involve a variety of interferences, and the training samples used to train the re-identification model cannot cover all possible interferences in real-world scenarios. This results in low robustness of the trained re-identification model, meaning that in some real-world scenarios, the accuracy of image regions belonging to the same object determined by the re-identification model is not high.

[0031] The object re-identification method provided in this application can improve upon the re-identification scheme by utilizing strategy design in the re-identification process to enhance the accuracy (or recognition performance) of identifying image regions belonging to the same object in different scenarios. This method can be applied to electronic devices, such as servers. Servers can be used for traffic management, processing videos captured by public safety cameras, etc., and this application does not limit this to specific types. After acquiring the first video captured by the camera, the electronic device can identify the first matching region of a specified object in the first video according to the object re-identification method provided in this application's embodiments. That is, from the object image region occupied by the object in the first video, the first matching region belonging to the specified object is determined, resulting in a highly accurate re-identification result.

[0032] See Figure 1 , Figure 1 A first flowchart of the object re-identification method provided in this application embodiment, the method may include the following steps:

[0033] S101: Obtain the video frame to be re-identified in the first video as the video frame to be identified.

[0034] S102: From the object image region occupied by the object in the video frame to be identified, determine the next object image region to be matched with the template image of the specified object, and use it as the current region to be matched.

[0035] S103: If the current region to be matched does not match the template image, add the current region to be matched to the first candidate region set, and return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0036] S104: If the current region to be matched matches the template image, add the current region to be matched to the first matching region set of the specified object, and add the first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set. Then return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0037] Based on the above processing, the video frames to be re-identified in the first video include the object image region occupied by the object. For the object image region whose current status needs to be determined (i.e., the current matching region), the electronic device first matches the object image region with the template image of the specified object. If the object image region does not match the template image, it is added to the first candidate region set, i.e., it is first used as the first candidate region. If the object image region matches the template image, it is added to the first matching region set, i.e., it is used as the first matching region of the specified object, and then matched with the first candidate region set. For the first candidate region that does not match the template image, the first matching region that matches the template image can be used to match the first candidate region again, to further filter the first candidate region and determine whether it belongs to the specified object, i.e., to identify the missed first matching regions. The final first matching region in the first matching region set is the object image region that the electronic device has determined to belong to the specified object by performing object re-identification on the first video. In this way, the probability of missed detection can be reduced, and more accurate re-identification results can be obtained.

[0038] Regarding step S101, a camera can be set up in a real-world scenario to capture video of that scenario. After capturing video frames, the camera can first cache the video frames in its own buffer space. When the number of cached video frames reaches a specified number, it sends a video acquisition command to the electronic device. After receiving the video acquisition command, the electronic device acquires each currently cached video frame in the camera's buffer space, thus obtaining the video captured by the camera. The total data volume of the specified number of video frames does not exceed the upper limit of the camera's buffer space. To reduce the probability of video frame loss caused by newly acquired video frames being overwritten by previously acquired video frames, the total data volume of the specified number of video frames can be 80% or 90% of the upper limit, etc., and this application does not impose any limitations on this. That is, the electronic device acquires one video segment from the camera each time, and there is an interval between acquiring two video segments.

[0039] In real-world scenarios, cameras can continuously capture video frames. Alternatively, cameras can normally remain in sleep mode; sensors (such as passive infrared sensors) can send a wake-up command to the camera after detecting an object passing within its field of view, causing the camera to wake up and capture video frames.

[0040] The video currently acquired by the electronic device for object re-identification is the first video.

[0041] In some embodiments, when the camera is continuously capturing video frames, the electronic device can first detect whether there is an object in each video frame of the currently acquired video; then, the video composed of the video frames containing the object is used as the first video. When the sensor wakes up the camera to capture video frames, the electronic device can directly use the currently acquired video as the first video.

[0042] The video frame to be identified is the video frame in the first video that needs to be re-identified. For example, the electronic device can directly use all video frames of the first video as the video frame to be identified.

[0043] In step S102, after determining multiple video frames to be identified, the electronic device can perform object detection on each video frame to determine the image region occupied by the object in the video frame; and extract the image region from the video frame to obtain the object image region occupied by the object in the video frame.

[0044] The designated object is the object that needs to be re-identified. As mentioned earlier, an indoor camera can be a home camera in a household, and the designated object can be a family member of that household. The home camera can be bound to an application on a user terminal (such as a mobile phone) used by family members; family members can upload their own template images in this application. In practical scenarios, there can be one or more designated objects, and one or more template images for a single designated object. This application does not limit the specific number of designated objects or the specific number of template images for each designated object. For ease of description, this application will subsequently use one designated object and one template image as an example.

[0045] The electronic device can identify an object image region from the extracted object image region that has not yet been matched with the template image of the specified object, and use this region as the current region to be matched. For example, the electronic device can randomly select an object image region that has not been matched with the template image of the specified object. Alternatively, the electronic device can extract the object image region occupied by the object in each video frame to be identified in the first video according to the position of each video frame to be identified; and determine the object image regions to be matched with the template image in the order of extraction of the object image regions.

[0046] In steps S103 and S104, after determining the current area to be matched, the electronic device can match the current area to be matched with the template image of the specified object.

[0047] For example, an electronic device can extract image features of the current region to be matched and the template image respectively, calculate the distance between the two extracted image features, and use the reciprocal of the distance as the similarity between the current region to be matched and the template image; if the similarity is greater than the similarity threshold, it is determined that the current region to be matched matches the template image; otherwise, it is determined that the current region to be matched does not match the template image.

[0048] For example, electronic devices can use feature-based matching technology to find the same feature points in the current region to be matched and the template image, so as to determine whether the current region to be matched matches the template image based on the feature points.

[0049] For example, electronic devices can use MI (Mutual Information) technology to calculate the mutual information between the current region to be matched and the template image based on the grayscale values ​​of pixels in the template image, so as to determine whether the current region to be matched matches the template image based on the mutual information.

[0050] In real-world scenarios, there are other ways to match the current region to be matched with the template image, and this application does not limit this to such methods.

[0051] If the current region to be matched does not match the template image, the current region to be matched is added to the first candidate region set, and the current region to be matched becomes the first candidate region. Then, the next object image region to be matched with the template image is determined from the aforementioned object image regions.

[0052] If the current region to be matched matches the template image, it means that the object in the current region to be matched is the specified object. Therefore, the current region to be matched is added to the first matching region set of the specified object. Accordingly, the current region to be matched is the first matching region of the specified object.

[0053] In real-world scenarios, a match between the current region to be matched and the template image only indicates similarity. Differences may still exist between them. Therefore, the first candidate region may match the current region to be matched; that is, the object in the first candidate region may be the same as the object in the current region to be matched, both being the specified object. Thus, when the first candidate region set is updated—that is, when a new first candidate region is added to the first candidate region set—the electronic device matches the newly added first candidate region (i.e., the current region to be matched) with the first candidate region set, i.e., matches it with the first candidate region in the current first candidate region set. The method by which the electronic device matches the current region to be matched with the first candidate region set is similar to the method described above for matching the current region to be matched with the template image; please refer to the relevant descriptions in the foregoing embodiments.

[0054] The first candidate region that matches the current region to be matched is added to the first matching region set of the specified object. That is, the first candidate region is removed from the current first candidate region set and added as the first matching region to the first matching region set. An object image region cannot belong to both the first candidate region set and the first matching region set simultaneously.

[0055] For example, the electronic device extracts 5 video frames to be identified from video 1, each of which includes an object image region; the object image region in the first extracted video frame to be identified is denoted as A_1, the object image region in the second extracted video frame to be identified is denoted as A_2, and so on, with the object image region in the fifth extracted video frame to be identified being denoted as A_5; the template image of object A is denoted as A_Moban.

[0056] In A_Moban, object A is wearing red clothing and its face is visible (it can be said to be facing the screen). In A_1, object A is wearing green clothing but its face is not visible (it can be said to be facing away from the screen). In A_2, object A is wearing green clothing and is facing the screen. When the electronic device uses A_1 as the current region to be matched, the object in A_1 differs significantly from object A in A_Moban, resulting in a low similarity between A_1 and A_Moban. Therefore, A_1 does not match A_Moban and is added to the first candidate region set. At this point, the first candidate region set includes A_1; the first matching region set for object A is empty.

[0057] Then, the electronic device selects A_2 as the current region to be matched. Since both the object in A_2 and object A in A_Moban are facing the screen, the difference between the object in A_2 and object A in A_Moban is small, and the similarity between A_2 and A_Moban is high. Therefore, A_2 matches A_Moban and is added to the first matching region set. At this point, the first candidate region set includes A_1; the first matching region set includes A_2. Because the first matching region set changes, the first matching region set is updated, and the electronic device can then match A_1 and A_2.

[0058] Since both object A in A_1 and object A in A_2 are wearing green clothing, and the differences between them are small, A_1 and A_2 have a high similarity. Therefore, A_1 and A_2 match, meaning the object in A_1 is also object A. The electronic device removes A_1 from the first candidate region set and adds it to the first matching region set. At this point, the first candidate region set is empty; the first matching region set includes A_2 and A_1. Thus, A_1 is successfully identified and corrected to belong to object A using the successfully matched A_2. Then, the electronic device uses A_3 as the current region to be matched. This process continues until all object image regions in the video frames to be identified in the first video are matched.

[0059] The above-described method of adding object image regions that do not match the template image (referred to as failed object bounding boxes) to the first candidate region set (referred to as adding to the candidate pool) and then matching the first matching regions added to the first matching region set with the first candidate regions in the first candidate region set (referred to as triggering successful sample matching) is called object re-identification with a candidate pool mechanism, or candidate mechanism optimization. After matching all object image regions in the video frames to be identified in the first video, all first matching regions in the first matching region set of the specified object, i.e., the identified object image regions belonging to the specified object in the first video, are the object re-identification results of the first video. This can be called the object re-identification result within a single segment based on the candidate pool mechanism.

[0060] The candidate pool mechanism can greatly correct the incorrectly identified target (i.e., the first candidate region that does not match the template image). When using the re-identification model to realize object re-identification and post-processing, there is no need to perform additional self-learning or clustering operations on the re-identification model, which reduces the training difficulty of the re-identification model.

[0061] In some embodiments, when a successful sample and candidate match is triggered, for each first candidate region in the current first candidate region set, the electronic device can combine multiple first matching regions in the current first matching region set and perform a joint match with that first candidate region. For example, the electronic device can calculate the similarity between each first matching region and the first candidate region, and calculate the average of the obtained similarities. If the average value is greater than a similarity threshold, the first candidate region is added to the first matching region set. This reduces the probability of missed detections and improves the accuracy of the obtained object re-identification results.

[0062] In some embodiments, the aforementioned step S101 may include the following steps:

[0063] Step A1: Obtain the first video.

[0064] Step A2: Extract multiple video frames from the first video based on the object's shape, as the video frames to be identified.

[0065] In this embodiment, the method by which the electronic device acquires the first video is described in the foregoing embodiments and will not be repeated here.

[0066] The objects can be set based on the business needs of the actual scenario, and this application does not limit the specific type of the objects. For example, when the camera is an indoor camera as described above, the objects can include people and animals; when the camera is an outdoor camera as described above, the objects can include vehicles.

[0067] The form of an object can be determined based on the specific type of the object, but this application does not limit the specific type of the object's form.

[0068] For example, if the object is a person, the person's form may include at least one of the following: the person's posture, orientation, degree of occlusion, and clothing color. The person's posture may include any of the following: standing, sitting, walking, running, and bending over; the person's orientation may include any of the following: facing forward, facing backward, facing to the side, and turning the face at a specified angle; the person's degree of occlusion may include any of the following: no occlusion, partial occlusion, severe occlusion, upper body occlusion, and lower body occlusion; the person's clothing color may include any of the following: red, blue, green, black, white, and multicolored.

[0069] For example, if the object is a vehicle, the vehicle's form may include at least one of the following: the vehicle's posture, the degree of occlusion, and the main body color. The vehicle's posture may include driving or parking; the degree of occlusion may include any of the following: no occlusion, partial occlusion, or severe occlusion; the main body color may include any of the following: red, blue, green, black, white, or multicolored.

[0070] For example, if the object is an animal (such as a cat), the cat's form may include at least one of the following: the cat's posture, degree of occlusion, and fur color. The cat's posture may include any of the following: standing, sitting, walking, and curled up; the cat's degree of occlusion may include any of the following: no occlusion, partial occlusion, severe occlusion, front half occlusion, and rear half occlusion; the cat's fur color may include any of the following: black, white, orange, gray-blue, and multicolored.

[0071] Furthermore, the electronic device can extract frames from the first video based on the object's shape to obtain the video frame to be identified. For example, the electronic device can extract the video frame to be identified based on an object shape frame extraction algorithm. The object shape frame extraction algorithm can be an inter-frame difference method, a shot boundary detection method, or a spectral clustering method, etc., and this application does not limit it to these methods.

[0072] Based on the above processing, by extracting frames from the first video, the number of video frames to be identified is reduced, thus reducing the number of object image regions that need to be matched subsequently, thereby reducing the time spent obtaining object re-identification results. Furthermore, extracting representative video frames from the first video based on the object's shape for object re-identification can reduce the aforementioned time consumption while avoiding a decrease in the accuracy of the obtained object re-identification results.

[0073] In some embodiments, step A2 may include the following steps:

[0074] Step B1: For each morphological dimension of the object, obtain the preset keywords used to describe the different forms of the object in that morphological dimension.

[0075] Step B2: For each video frame in the first video, perform image-text matching between the video frame and each preset keyword to obtain the matching degree between the video frame and each preset keyword.

[0076] Step B3: Select the preset keywords that match the video frame with a matching degree not less than the matching degree threshold as the keywords to be extracted from the corresponding video frame.

[0077] Step B4: If the number of identified keywords to be extracted is not less than a preset number, for each keyword to be extracted, select one video frame from the video frames corresponding to that keyword as the representative frame of that keyword; select a preset number of representative frames from the representative frames of each keyword to be extracted as the video frames to be identified extracted from the first video.

[0078] Step B5: If the number of identified keywords to be extracted is less than the preset number, for each keyword to be extracted, select at least one video frame from the video frames corresponding to that keyword to obtain the preset number of video frames, which are used as the video frames to be identified extracted from the first video.

[0079] In this embodiment, extracting multiple video frames to be identified from the first video based on keywords can be called intelligent frame extraction. A preset keyword is used to describe a possible form of an object in a morphological dimension. That is, a corresponding keyword can be set according to the possible forms of an object in a real scene. These multiple keywords can be recorded in the Prompt table. For example, based on the aforementioned example, when the object is a person, the object's form can include the person's posture, orientation, degree of occlusion, and clothing color. Accordingly, the object's morphological dimension can include the human posture dimension, human orientation dimension, degree of occlusion dimension, and clothing color dimension.

[0080] A person's posture can include any of the following: standing, sitting, walking, running, and bending over; correspondingly, the preset keywords used to describe different forms of a person in the human posture dimension can include: "posture-standing": "a person standing straight"; "posture-sitting": "a person sitting on a table"; "posture-walking": "a person walking normally"; "posture-running": "a person running very fast"; "posture-bending": "a person bending over"; "posture-squatting": "a person squatting down".

[0081] The orientation of a person can include any of the following: facing forward, facing backward, facing to the side, and the face rotated at a specified angle; correspondingly, the preset keywords used to describe different forms of a person in the human-shaped orientation dimension can include: "Orientation - Facing forward": "A person faces forward, with the whole face visible"; "Orientation - Facing to one side": "A person faces to the side, side view"; "Orientation - Facing backward": "A person faces backward, with their back to the camera"; "Orientation - 45-degree angle": "A person faces at a 45-degree angle".

[0082] The degree of occlusion of a person can include any of the following: no occlusion, partial occlusion, severe occlusion, upper body occlusion, and lower body occlusion; correspondingly, the preset keywords used to describe the different forms of a person under the dimension of occlusion degree can include: "Occlusion-None": "A person is completely visible and there is no occlusion"; "Occlusion-Partial": "A person is partially occluded by an object"; "Occlusion-Severe": "A person is severely occluded, and most of them are hidden"; "Occlusion-Upper Body": "A person whose upper body is occluded"; "Occlusion-Lower Body": "A person whose lower body is occluded".

[0083] The main color of a person's clothing can include any of the following: red, blue, green, black, white, and multicolored; correspondingly, the preset keywords used to describe different forms of a person in the dimension of clothing color can include: "red-clothes": "a person wearing red clothes"; "blue-clothes": "a person wearing blue clothes"; "green-clothes": "a person wearing green clothes"; "black-clothes": "a person wearing black clothes"; "white-clothes": "a person wearing white clothes"; "multicolored-clothes": "a person wearing multicolored clothes".

[0084] In real-world scenarios, the form dimension of an object can also be a combination dimension obtained by combining the above form dimensions. For example, preset keywords used to describe different forms of a person under the combination dimension can include: "Facing forward - Standing": "A person is facing forward and standing straight"; "Facing to one side - Walking": "A person is facing to the side and walking"; "Facing backward - Running": "A person is facing backward and running"; "Sitting - Partial": "A person is sitting on a table and is partially obscured"; "Standing - Red": "A person is standing straight and wearing red clothes"; "Facing forward - Walking - Blue": "A person is facing forward and walking, and wearing blue clothes"; "Facing to one side - Standing - Partial": "A person is facing to the side, standing straight, and is partially obscured by an object."

[0085] In real-world scenarios, there are more preset keywords than those listed above, and this application does not limit this to them.

[0086] After obtaining the preset keywords, for each video frame in the first video, the electronic device can perform image-text matching between the video frame and each preset keyword to obtain the matching degree between the video frame and each preset keyword. For example, the video frame and each preset keyword can be input into a pre-trained image-text matching model (also known as a multimodal model) to obtain multiple matching degrees output by the image-text matching model; a matching degree represents the similarity (also known as the Prompt similarity) between the form of the object described by a preset keyword and the form of the object in the video frame. The electronic device can use preset keywords with a matching degree of not less than the matching degree threshold as the frame-extracting keywords to describe the form of the object in the video frame, i.e., the frame-extracting keywords corresponding to the video frame. The matching degree threshold can be set based on the needs of the actual scenario, such as 0.5. The image-text matching model can be CLIP (Contrastive Language-Image Pre-training), ALING (A Large-scale Image and Noisy-Text Embedding), or BLIP (Bootstrapping Language-Image Pre-training), etc., and this application does not limit it to any particular model.

[0087] A single video frame can have multiple keywords to be extracted; these keywords can belong to different or the same morphological dimensions. For example, if the video frame includes two objects, one standing upright and the other squatting, then the keywords to be extracted for this video frame can include the aforementioned "posture-standing": "person standing upright" and "posture-squatting": "person squatting". If the video frame includes one object, and this object is squatting and wearing red clothing, then the keywords to be extracted for this video frame can include the aforementioned "posture-squatting": "person squatting" and "red-clothing": "a person wearing red clothing". The actual scenarios may involve more than these, and this application does not limit this.

[0088] In real-world scenarios, there might be situations where the match degree between a preset keyword and each video frame of the first video is less than a matching degree threshold. Therefore, after obtaining the keywords to be extracted for each video frame, the electronic device needs to count the number of video frames containing the corresponding keywords.

[0089] If the number of identified keywords to be extracted is not less than a preset number, it indicates that the objects in the first video have diverse forms, and the electronic device can select video frames from the first video that include as many diverse object forms as possible. The preset number is set based on the needs of the actual scenario; for example, in order to perform uniform processing on each video acquired from the camera, the electronic device can set the number of video frames in the video that the electronic device needs to process for each video (i.e., the preset number). Accordingly, after acquiring a video each time, the electronic device will extract a preset number of video frames from the video, and then perform object re-identification on the extracted preset number of video frames; and use the object re-identification result obtained based on the preset number of video frames as the object re-identification result for the video. The preset number can be 10, 20, or 30, and this application does not limit it.

[0090] For each keyword to be extracted, the electronic device can select one video frame from the video frames corresponding to that keyword as the representative frame. For example, the electronic device can randomly select one video frame from the video frames corresponding to that keyword as the representative frame. Alternatively, the electronic device can select the video frame with the highest matching degree to the keyword from the video frames corresponding to that keyword, that is, the video frame whose object shape best matches the shape of the object described by the keyword, as the representative frame.

[0091] Then, a preset number of representative frames are selected from the representative frames of each keyword to be extracted, as the video frames to be identified extracted from the first video. For example, the electronic device may randomly select a preset number of representative frames from these multiple representative frames as the video frames to be identified. Alternatively, the electronic device may select the top preset number of representative frames with the highest matching degree as the video frames to be identified.

[0092] In real-world scenarios, electronic devices select different video frames to be identified. For example, the video frame with the highest matching degree corresponding to different keywords to be extracted may be the same video frame. In this case, the electronic device can use this video frame as the representative frame for any of the above keywords to be extracted; for other keywords to be extracted, the video frame with the second highest matching degree is used as the representative frame for the other keywords to be extracted.

[0093] If the number of identified keywords to be extracted is less than the preset number, it indicates that the morphological changes of objects in the first video are relatively few. In order to obtain as many diverse object morphologies as possible, for each keyword to be extracted, the electronic device must select at least one video frame from the video frames corresponding to that keyword; and the total number of video frames selected by the electronic device is the aforementioned preset number, and the selected video frames can be used as video frames to be identified. For example, if the preset number is 10 and the number of keywords to be extracted is 5, then for each keyword to be extracted, 2 video frames can be randomly selected from the video frames corresponding to that keyword; or, the video frames with the highest matching degree can be selected from the video frames corresponding to any 4 keywords to be extracted, and then 6 video frames can be randomly selected from the video frames corresponding to the remaining 1 keyword to be extracted. In actual scenarios, when the number of identified keywords to be extracted is less than the preset number, the electronic device may select the preset number of video frames to be identified in other ways than this, and this application does not limit this.

[0094] Based on the above processing, the electronic device can extract video frames (also known as keyframes) from the first video that contain a high degree of diversity in the morphological forms of the objects. The high diversity in the morphological forms of the objects in the video frames can be described as increasing the diversity of single REID recognition samples, or, in the case of people, increasing the diversity of human figures in the extracted human figure keyframes. In this way, representative and differentiated video frames containing objects can be obtained, avoiding the situation where the extracted video frames contain objects with similar forms. This reduces the time spent obtaining object re-identification results while maintaining the accuracy of the obtained results.

[0095] In some embodiments, when the current region to be matched matches the template image, the electronic device can match the current region to be matched with a first candidate region set; all first candidate regions in the first candidate region set that match the current region to be matched are added to the first matching region set. Then, the next object image region to be matched with the template image is determined from the object image regions. In this way, the complexity of the method can be reduced, the computational resources required to perform the object re-identification method can be reduced, and the efficiency of obtaining the object re-identification result can be improved.

[0096] In some embodiments, adding the first candidate region that matches the current region to be matched in the first candidate region set to the first matching region set in the aforementioned step S104 may include the following steps:

[0097] Step C1: From the first candidate region set, determine the next first candidate region to be matched with the current region to be matched, and use it as the current region to be corrected.

[0098] Step C2: If the current region to be corrected does not match the current region to be matched, return to the previous step C1.

[0099] Step C3: If the current region to be corrected matches the current region to be matched, add the current region to be corrected to the first matching region set, and determine the current region to be corrected as the new current region to be matched; return to execute the aforementioned step C1;

[0100] Step C4: Take the first matching region previously added to the first matching region set as the new current region to be matched; return to execute the aforementioned step C1.

[0101] In this embodiment, the electronic device can randomly select a first candidate region from the first candidate region set that has not yet been matched with the current region to be matched, as the current region to be corrected. Alternatively, since the electronic device only matches one object image region with the template image at a time, that is, only one object image region is added to the first candidate region set at a time, there is also an order among the first candidate regions added to the first candidate region set. Therefore, the electronic device can select the earliest added first candidate region among the first candidate regions that has not yet been matched with the current region to be matched, according to the order in which the first candidate regions were added to the first candidate region set, as the current region to be corrected.

[0102] After determining the current region to be corrected, the electronic device can match the current region to be corrected with the current region to be matched (i.e., the newly determined first matching region). The method by which the electronic device matches the current region to be corrected with the current region to be matched is similar to the method described above for matching the current region to be matched with the template image, and can be referred to the relevant description in the previous embodiments, which will not be repeated here.

[0103] If the current region to be corrected does not match the current region to be matched, the electronic device continues to use the current region to be matched to match other first candidate regions, that is, it continues to determine the next first candidate region to be matched with the current region from the set of first candidate regions.

[0104] If the current region to be corrected matches the current region to be matched, it means that the current region to be corrected is similar to the newly determined first matching region. Therefore, it can be determined that the object in the current region to be corrected is also the specified object. The current region to be corrected is then deleted from the first candidate region set and added to the first matching region set. At this point, the first matching region set is updated; the electronic device determines the newly determined first matching region of the specified object (i.e., the current region to be corrected) as the new current region to be matched; only one current region to be matched can exist at any given time. Correspondingly, the current region to be matched with the current region to be corrected is used as the first matching region previously added to the first matching region set.

[0105] Then, the electronic device determines the next first candidate region from the first candidate region set to be matched with the current region to be matched. This process is repeated until the first matching region set is no longer updated. That is, each time the first matching region set is updated, it triggers the matching of the newly added first matching region to the first candidate region set with the first candidate region in the current first candidate region set. The first candidate region that matches the newly added first matching region is added to the first matching region set (this can be called object re-identification within a single segment of the first video based on a candidate pool mechanism).

[0106] For example, the object is a person, and the specified object is person A; the template image of person A is denoted as A_Moban; in A_Moban, person A is wearing red clothes, his body is not obscured, he is facing the screen, and he is standing. The object image regions occupied by the object in the multiple video frames to be identified in the first video are a total of 10, which are denoted as A1, A2, A3, ..., A10 respectively.

[0107] The electronic device first matches A1 with A_Moban. If the person in A1 is wearing green clothes, their lower body is obscured by a coffee table, they are facing away from the screen, and they are standing; because the shape of the person in A1 is significantly different from the shape of person A in A_Moban, the similarity between A1 and A_Moban is low, so A1 and A_Moban do not match. The electronic device can create a candidate pool queue (i.e., the aforementioned first candidate region set), denoted as candidate_person_list, and then add A1 to the candidate pool queue, that is, add A1 to the first candidate region set. At this time, the candidate pool queue can be denoted as candidate_person_list=[A1].

[0108] Then, the electronic device matches A2 with A_Moban. If the person in A2 is wearing green clothes, their lower body is obscured by a coffee table, facing the screen, and sitting on a sofa; because the shape of the person in A2 differs significantly from the shape of person A in A_Moban, A2 and A_Moban have a low similarity, so A2 does not match A_Moban. The electronic device can add A2 to the candidate pool queue. At this time, the candidate pool queue can be denoted as candidate_person_list=[A1,A2]. Similarly, the electronic device continues to match the object image region with A_Moban. Assuming that A3 and A4 do not match A_Moban, the electronic device adds A4 to the candidate pool queue, and the candidate pool queue can now be denoted as candidate_person_list=[A1,A2,A3,A4].

[0109] Continue matching A5 with A_Moban. If the person in A5 is wearing green clothes, their body is not obscured, they are facing the screen, and they are standing; since the difference between the shape of the person in A5 and the shape of person A in A_Moban is small, the similarity between A5 and A_Moban is high, and it can be determined that A5 matches A_Moban. The electronic device can create a successful identity recognition list (i.e., the first matching area set of person A), denoted as A_success_list, and add A5 to the successful identity recognition list, that is, add A5 to the first matching area set of person A. At this time, the successful identity recognition list can be denoted as A_success_list=[A5]; at this time, the candidate pool queue is candidate_person_list=[A1,A2,A3,A4].

[0110] Because the first matching region set of character A is updated, the newly added first matching region (i.e., A5) is used to match each first candidate region (i.e., A1-A4) in the candidate pool.

[0111] At this point, A1 is the first image region added to the candidate pool queue. The electronic device first matches A1 with A5. Assuming A1 matches A5, A1 is added to the first matching region set, which means removing A1 from the candidate pool queue and adding A1 to the successful identity recognition list. At this point, the successful identity recognition list is A_success_list=[A5,A1]; and the candidate pool queue is candidate_person_list=[A2,A3,A4].

[0112] Because the first matching region set of character A is updated, the newly added first matching region (i.e., A1) in the first matching region set is used to match each first candidate region (i.e., A2-A4) in the candidate pool.

[0113] At this point, A2 is the first image region added to the candidate pool queue. The electronic device first matches A1 with A2. Assuming A1 and A2 match, the electronic device adds A2 to the current first matching region table, that is, removes A2 from the candidate pool queue and adds A2 to the successful identity recognition list. At this point, the successful identity recognition list is A_success_list=[A5,A1,A2]; the candidate pool queue is candidate_person_list=[A3,A4].

[0114] Because the first matching region set for character A is updated, the newly added first matching region (A2) is matched against the first candidate regions (A3 and A4) in the candidate pool. This continues. If A2 does not match A3, and A2 does not match A4, then there are no first candidate regions not matched with A2. The matching process for A2 in this round is complete. The electronic device then uses the previously added first matching region (A1) as the new region to be matched, and then matches A1 with A3. This cycle continues until the first matching region set is no longer updated. At this point, every first matching region in the first matching region set has been matched against every first candidate region in the current first candidate region set. The electronic device then continues matching A6 with A_Moban.

[0115] This process continues until matching of A1-A10 is complete, and both the candidate pool queue and the successful identity recognition list are no longer updated. At this point, the re-identification process for the first video ends. The object image regions added to the successful identity recognition list at this time are those that belong to person A, as determined by the first video itself; the object image regions added to the candidate pool queue at this time are those that do not belong to person A, as determined by the first video itself.

[0116] Based on the above processing, each time the first matching region set is updated, object re-identification within a single segment of the first video is triggered based on the candidate pool mechanism. This ensures that each first candidate region is matched with as many first matching regions as possible, reducing the probability of missed detections and further improving the accuracy of the identified image regions belonging to the specified objects, thus further improving the accuracy of the obtained re-identification results.

[0117] In some embodiments, there are multiple specified objects; a matching region in the matching region set of a specified object matches the template image of the specified object, or a matching region in the matching region set of the specified object matches any other matching region in the matching region set of the specified object.

[0118] In this embodiment, when there are multiple specified objects, each specified object corresponds to a template image and a first matching region set. When matching the current region to be matched with the template image, the current region to be matched can be matched with the template image of each specified object separately.

[0119] If the current region to be matched does not match any template image of any specified object, the current region to be matched is added to the first candidate region set. If the current region to be matched matches a template image of a specified object, the current region to be matched is added to the first matching region set of that specified object. An update to the first matching region set of any specified object will trigger a matching process between the latest first matching region of that specified object and each first candidate region. The first candidate region that matches the latest first matching region of that specified object will be added to the first matching region set of that specified object. The way the electronic device adds the first candidate region that matches the latest first matching region of that specified object to the first matching region set is similar to the way it adds the first candidate region that matches the current region to be matched to the first matching region set, and can be referred to the relevant description in the foregoing embodiments, which will not be repeated here.

[0120] For example, similar to the previous example, the specified object may include person A and person B; the template image of person A is denoted as A_Moban, and the template image of person B is denoted as B_Moban; the object image regions occupied by the objects in multiple keyframes of the first video total 20, which are denoted as A1, B1, A2, B2, A3, B3, ..., A10, B10 respectively.

[0121] The electronic device first matches A1 with A_Moban, and then matches A1 with B_Moban. Assuming that A1 does not match A_Moban and also does not match B_Moban, a candidate pool queue is created, denoted as candidate_person_list, and A1 is added to the candidate pool queue, resulting in candidate_person_list=[A1].

[0122] Match B1 with A_Moban, and match B1 with B_Moban. Assuming B1 does not match A_Moban but matches B_Moban, create a successful identification list for person B (hereinafter referred to as success list B), denoted as B_success_list, and then add B1 to success list B, resulting in B_success_list=[B1].

[0123] At this point, the success list B is updated, and B1 is matched against A1. Assuming A1 does not match B1, the matching process for B1 in this round is complete. The next step is to match A2 against A_Moban, and then against B_Moban. Assuming A2 matches A_Moban but not B_Moban, a success list for identifying person A (hereinafter referred to as success list A) is created, denoted as A_success_list. Then, A2 is added to success list A, resulting in A_success_list = [A2].

[0124] At this point, the successful list A is updated, and A2 is matched against A1. Assuming A1 and A2 do not match, the matching process for A2 in this round is complete. The next step is to match B2 against A_Moban, and then against B_Moban. Assuming B2 does not match either A_Moban or B_Moban, B2 is added to the candidate pool queue, resulting in candidate_person_list=[A1,B2].

[0125] The system matches A3 with A_Moban and A3 with B_Moban. If A3 matches A_Moban but not B_Moban, A3 is added to the success list A, resulting in A_success_list = [A2, A3]. Success list A is then updated, and the electronic device matches A3 with A1. If A3 matches A1, A1 is added to success list A, resulting in A_success_list = [A2, A3, A1]. Success list A is then updated, and the electronic device matches A1 with B2. If A1 does not match B2, the system continues to match A3 with B2. If A3 also does not match B2, the electronic device continues to match B3 with A_Moban and B3 with B_Moban. This process continues until the matching of A1-A10 and B1-B10 is completed, and the candidate pool queue, success list A, and success list A are no longer updated. At this point, the re-identification process for the first video ends. The object image regions added to the success list A at this point are those belonging to person A, as determined by the first video itself. The object image regions added to the success list B at this point are those belonging to person B, as determined by the first video itself. The object image regions added to the candidate pool queue at this point are those that do not belong to person A or person B, as determined by the first video itself.

[0126] Based on the above processing, when there are multiple specified objects, the electronic device can determine the first matching region of each specified object according to the object re-identification method provided in the embodiments of this application, thereby meeting the business needs of actual scenarios and improving the applicability of the object re-identification method.

[0127] In some embodiments, to further improve the accuracy of the obtained object re-identification results, after the electronic device obtains the object re-identification result within a single segment based on the candidate pool mechanism, it can also perform inter-segment error correction based on the object re-identification results of adjacent segments. This method can be called pedestrian re-identification with an inter-segment sample error correction mechanism. Specifically, after completing the matching of all object image regions in the video frame to be identified in the first video, the method further includes the following steps:

[0128] Step D1: Obtain the second matching region set of the second video.

[0129] Step D2: For each first candidate region in each first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, add the first candidate region to the first matching region set.

[0130] In this embodiment, the electronic device can match the object image regions in the video frames to be identified in the video according to the object re-identification method provided in the foregoing embodiments each time it acquires video from the camera. Therefore, the earlier the electronic device acquires a video, the earlier it completes the matching of all object image regions in the video frames to be identified. For ease of description, the time when the matching of all object image regions in the keyframes of the first video is completed will be referred to as the first moment, and the time when the matching of all object image regions in the keyframes of the second video is referred to as the second moment.

[0131] The relationship between the first and second moments is not limited in the embodiments of this application. For example, the second moment may be earlier than the first moment, or the second moment may be later than the first moment. The first and second moments may be adjacent, that is, the first video and the second video are videos acquired by the electronic device from the camera at two different times; or, there may be a moment between the first and second moments when the matching of all object image regions in the video frames to be identified in other videos is completed, that is, the electronic device may acquire other videos between the moment when it acquires the first video and the moment when it acquires the second video.

[0132] For each candidate region in the first candidate region set, if a second matching region exists in the second matching region set that matches the first candidate region, the first candidate region is added to the first matching region set. This can be called correcting the first matching region set using the second matching region set, or correcting the first video using the second video. In other words, it updates both the first candidate region set and the first matching region set.

[0133] The second matching region set of the second video can be obtained by the electronic device after acquiring the second video from the camera, matching the object image regions occupied by the objects in the video frames to be identified in the second video in a manner similar to that used to acquire the first matching region set of the first video. Specifically, the process involves: acquiring the video frames to be re-identified in the second video; determining the next matching region to be matched with the template image from the object image regions occupied by the objects in the video frames to be identified; if the current matching region does not match the template image, adding the current matching region to the second candidate region set, and continuing to determine the next matching region to be matched with the template image from the object image regions in the second video; if the current matching region matches the template image, adding the current matching region to the second matching region set of the specified object, and adding the second candidate regions in the second candidate region set that match the current matching region to the second matching region set, and then continuing to determine the next matching region to be matched with the template image of the specified object from the object image regions in the second video, until the second matching region set is no longer updated, thus obtaining the second matching region set of the second video.

[0134] Alternatively, the electronic device can also perform error correction on the second video based on other videos, and use the corrected second matching region set as the acquired second matching region set. The method by which the electronic device performs error correction on the second video based on other videos is similar to the method by which it performs error correction on the first video based on the second video, and can be found in the description of relevant embodiments.

[0135] When the electronic device performs error correction between segments of the first video and the second video, object re-identification within a single segment of the second video, as well as error correction between segments of other videos and the second video, have been completed.

[0136] Furthermore, the electronic device can use the second matching region set to correct the error in the first matching region set.

[0137] In one implementation, the electronic device can determine the next first candidate region to be matched with the second matching region set from the first candidate region set, and use this as the current region to be processed. If the current region to be processed matches any of the second matching regions, the current region to be processed is added to the first matching region set. Then, the device continues to determine the next first candidate region to be matched with the second matching region set from the first candidate region set. This reduces the complexity of the method, decreases the computational resources required to execute the object re-identification method, and improves the efficiency of obtaining object re-identification results.

[0138] In another implementation, step D2 may include the following steps:

[0139] Step E1: From the first candidate region set, determine the next first candidate region to be matched with the second matching region set, as the current region to be processed.

[0140] Step E2: If none of the second matching regions in the second matching region set match the current region to be processed, return to step E1.

[0141] Step E3: If the current region to be processed matches any of the second matching regions in the second matching region set, add the current region to be processed to the first matching region set, and determine the current region to be processed as the new current region to be matched; add the first candidate regions in the first candidate region set that match the current region to be matched to the first matching region set; return to the step of determining the next first candidate region to be matched with the second matching region set from the first candidate region set.

[0142] In this implementation, the electronic device can determine the next first candidate region to be matched with the second matching region set from the first candidate region set in a manner similar to that described above, which determines the next object image region to be matched with the template image of the specified object from the object image regions occupied by the object in multiple video frames to be identified. This will not be elaborated here.

[0143] If none of the second matching regions in the second matching region set match the current region to be processed, the electronic device continues to select the next first candidate region from the first candidate region set to match the current second matching region.

[0144] If the current region to be processed matches any of the second matching regions, it means that the second matching region is similar to the current region to be processed, and the objects in the current region to be processed are the same as the objects in the second matching region, both being the specified objects. Therefore, the current region to be processed can be added to the first matching region set.

[0145] Since the first matching region set is updated at this time, the electronic device can add the first candidate regions that match the current region to be matched in the first candidate region set to the first matching region set in a similar manner as described above. Using the current first candidate regions and the current first matching regions, the electronic device can perform object re-identification within a single segment of the first video based on the candidate pool mechanism to update the first matching region set. The electronic device can continue to update the first matching region set according to the aforementioned steps C1-C3 until the first matching region set is no longer updated.

[0146] Then, the electronic device can continue to determine the next first candidate region to be matched with the second matching region set from the first candidate region set, and continue to update the first matching region set based on the second matching region set. This process is repeated until there are no more first candidate regions that are not matched with the second matching region set. The error correction of the first matching region set using the second matching region set is then complete.

[0147] Based on the above processing, after each update of the first matching region set based on inter-segment error correction, object re-identification within a single segment of the first video is triggered using a candidate pool mechanism. This ensures that each first candidate region is matched with as many first matching regions as possible, further improving the accuracy of the re-identification results.

[0148] In some embodiments, to further reduce the complexity of the method, reduce the computational resources required to perform the object re-identification method, and improve the efficiency of obtaining object re-identification results, the electronic device may use only the second matching region set to correct the first matching region set.

[0149] In some embodiments, to further improve the accuracy of the obtained object re-identification results, the electronic device may also use the first matching region set to correct the second matching region set. Specifically, before step D2, the method may further include the following steps: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, add the second candidate region to the second matching region set. Alternatively, after step D2, the method may further include the following steps: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, add the second candidate region to the second matching region set.

[0150] In this embodiment, the method of using the first matching region set to correct the second matching region set is similar to the aforementioned method of using the second matching region set to correct the first matching region set. That is, the electronic device determines the next second candidate region from the second candidate region set that will be matched with the first matching region set, and uses this as the current region to be processed. If none of the first matching regions in the first matching region set match the current region to be processed, the next second candidate region to be matched with the first matching region set is determined from the second candidate region set. If the current region to be processed matches any of the first matching regions, the current region to be processed is added to the second matching region set, and the current region to be processed is designated as the new current region to be matched. The second matching region set is updated in a manner similar to steps C1-C3 described above, until the second matching region set is no longer updated. Then, the next second candidate region to be matched with the current first matching region is determined from the second candidate region set.

[0151] Using a later-acquired video to correct an earlier-acquired video is called forward error correction; using an earlier-acquired video to correct a later-acquired video is called reverse error correction. Performing forward and reverse error correction on these two videos is called inter-segment mutual error correction. During this process, after the matching region set of either video is updated, object re-identification within a single segment of that video can be triggered based on a candidate pool mechanism to update the matching region set of that video. Once the matching region set of that video stops updating, inter-segment mutual error correction continues.

[0152] In other words, the electronic device can first use the first video to correct errors in the second video; then use the corrected second video to correct errors in the first video. Alternatively, it can first use the second video to correct errors in the first video; then use the corrected first video to correct errors in the second video. Regardless of which video's matching region set is updated, it can trigger object re-identification within a single segment of that video based on a candidate pool mechanism. This can further improve the accuracy of the obtained object re-identification results.

[0153] In some embodiments, the first video is the most recently captured video, and the second video includes videos captured within a preset duration range prior to the first video.

[0154] In this embodiment, the second moment is earlier than the first moment; the second video can be one or more. The preset duration range can be set based on the business needs of the actual scenario. Because when the preset duration range is large, the probability that the objects in both the first and second videos are the same specified object is low, and if the objects included in the first and second videos are different, mutual error correction between the first and second videos cannot be performed; furthermore, when the preset duration range is large, the number of second videos may be large, which will lead to an excessive number of object image regions needing to be matched when performing mutual error correction between segments of the first and second videos, thus reducing the efficiency of object re-identification results. Moreover, since there is a time interval between the electronic device acquiring the two video segments, when the preset duration range is small, the second video may not exist, and therefore, it is impossible to use the second video to correct the first video. Therefore, the preset error correction duration is usually not set too large or too small. In actual scenarios, the preset error correction duration can be set to 2 minutes, 5 minutes, or 10 minutes, etc., and this application does not limit this.

[0155] If the camera captures only one video within a preset duration before the first video, the second video will also be a single video. The electronic device can directly perform cross-correction between the first and second videos according to the aforementioned steps D1-D2.

[0156] If the camera captures multiple videos within a preset time range prior to the first video, then there will be multiple second videos. For each second video, before performing mutual error correction between the first and second videos, the electronic device has already matched the object image region occupied by the object in the video frame to be identified in the second video with the template image in a manner similar to the aforementioned steps S101-S104 (this can be called intra-segment matching); and has also performed error correction on the second matching region set of the second video using the videos captured within the preset time range prior to the second video (this can be called inter-segment error correction).

[0157] The electronic device can sequentially use each of the multiple second videos to correct errors in the first video, according to the order of their respective second moments. After correcting the first video using the second videos, it performs object re-identification within a single segment of the first video based on a candidate pool mechanism to update the current first matching region of the first video. Then, it uses the first video to sequentially correct errors in each of the second videos.

[0158] Alternatively, the electronic device can use the second video, corresponding to the earliest second moment of each of the multiple second videos, as the third video currently used for error correction of the first video, according to the order of the second moments of each second video. Based on the second matching region set of the third video, the first matching region set is updated, and then object re-identification within a single segment of the first video is performed using a candidate pool mechanism to update the first matching region set. Then, the second matching region set of the third video is updated based on the first matching region set; again, object re-identification within a single segment of the third video is performed using the candidate pool mechanism to update the second matching region set of the third video. Then, the second video corresponding to the second later second moment is used as the fourth video currently used for error correction of the first video. Based on the second matching region set of the fourth video, the first matching region set is updated, and then object re-identification within a single segment of the first video is performed using the candidate pool mechanism to update the first matching region set. Then, the second matching region set of the fourth video is updated based on the first matching region set; again, object re-identification within a single segment of the fourth video is performed using the candidate pool mechanism to update the second matching region set of the fourth video. Therefore, the second matching region set of the fourth video is first updated based on the second matching region set of the third video; then the second matching region set of the third video is updated based on the second matching region set of the fourth video. Alternatively, the second matching region set of the third video can be updated first based on the second matching region set of the fourth video; then the second matching region set of the fourth video can be updated based on the second matching region set of the third video.

[0159] This process is repeated, using the multiple second videos to correct the first video, and the multiple second videos can also correct each other, until the second matching region set of the multiple second videos stops updating, and the first matching region set also stops updating.

[0160] For example, the specified objects may include person A and person B; the template image of person A is denoted as A_Moban, and the template image of person B is denoted as B_Moban; the Xth video acquired by the electronic device is denoted as KX; each KX includes 20 object image regions, A1-A10 and B1-B10, which are denoted as KX_A1, KX_A2, ..., KX_A10, KX_B1, KX_B2, ..., KX_B10. When no inter-segment error correction is performed, the object re-identification result obtained for KX is denoted as KX recognition result, which includes the candidate region set of KX (denoted as KX_candidate_person_list), the matching region set for person A (denoted as KX_A_success_list), and the matching region set for person B (denoted as KX_B_success_list).

[0161] When the electronic device acquires video K1, K1 is the first video. The recognition results of K1 include: K1_A_success_list=[K1_A1,K1_A3,K1_A5,K1_A7]; K1_B_success_list=[K1_B1,K1_B3,K1_B5,K1_B7]; K1_candidate_person_list=[K1_A2,K1_A4,K1_A6,K1_A8,K1_A9,K1_A10,K1_B2,K1_B4,K1_B6,K1_B8,K1_B9,K1_B10]. At this time, the electronic device has not yet acquired the recognition results of the second video, so no inter-segment error correction is triggered.

[0162] When the electronic device acquires video K2, the process of acquiring the recognition result of K1 based on the template image has been completed. K1 is the second video, and K2 is the first video. Based on the candidate pool mechanism, object re-identification is performed within a single segment of K2 to obtain the K2 recognition result, including: K2_A_success_list=[K2_A2,K2_A4,K2_A6,K2_A8]; K2_B_success_list=[K2_B2,K2_B4,K2_B6,K2_B8]; K2_candidate_person_list=[K2_A1,K2_A3,K2_A5,K2_A7,K2_A9,K2_A10,K2_B1,K2_B3,K2_B5,K2_B7,K2_B9,K2_B10]. At this point, the electronic device has acquired the recognition results of K1 and K2, triggering error correction between the segments of K1 and K2. From K2_candidate_person_list, determine the next candidate region (i.e., K2_A1) to match the K1 recognition result, and use it as the current region to be processed. Match K2_A1 with each matching region in K1_A_success_list and K1_B_success_list to obtain the similarity between K2_A1 and each matching region; determine the matching region with the highest similarity to K2_A1 (let's assume it's K1_A3). If the similarity between K2_A1 and K1_A3 is greater than the similarity threshold, remove K2_A1 from K2_candidate_person_list and add K2_A1 to K2_A_success_list. K2_A_success_list is updated, triggering object re-identification within a single segment of K2 based on the candidate pool mechanism to update the K2 recognition result. The electronic device can update the K2 recognition result according to the aforementioned steps C1-C3 until the K2 recognition result no longer changes. Assuming that the object re-identification within a single segment of K2 fails to update the K2 identification result during the candidate pool mechanism, the next candidate region (K2_A3) to match the K1 identification result is determined from the K2_candidate_person_list, and this becomes the current region to be processed. K2_A3 is then matched against each matching region in K1_A_success_list and K1_B_success_list to obtain the similarity score between K2_A3 and each matching region. The matching region with the highest similarity score to K2_A3 (let's say K1_B3) is then determined. If the similarity score between K2_A3 and K1_B3 is not greater than the similarity threshold, the K2 identification result will not be updated.This process is repeated until there are no matching regions in K2_candidate_person_list that are not matched against K1_A_success_list and K1_B_success_list. The process of correcting K2 using K1 is then complete.

[0163] Then, the current recognition result of K2 is used to correct K1. Assume that the current recognition result of K2 includes: K2_A_success_list=[K2_A2,K2_A4,K2_A6,K2_A8,K2_A1,K2_A5,K2_A3]; K2_B_success_list=[K2_B2,K2_B4,K2_B6,K2_B8,K2_B1,K2_B5,K2_B3]; K2_candidate_person_list=[K2_A7,K2_A9,K2_A10,K2_B7,K2_B9,K2_B10]. From K1_candidate_person_list, the next candidate region (i.e., K1_A2) that matches the current recognition result of K2 is determined as the current region to be processed. Match K1_A2 with each matching region in K2_A_success_list and K2_B_success_list to obtain the similarity between K1_A2 and each matching region; determine the matching region with the highest similarity to K1_A2 (let's assume it's K2_A7). If the similarity between K1_A2 and K2_A7 is greater than the similarity threshold, remove K1_A2 from K1_candidate_person_list and add K1_A2 to K1_A_success_list. Update K1_A_success_list, triggering object re-identification within a single segment of K1 based on the candidate pool mechanism to update the K1 identification result. The electronic device can update the K1 identification result according to the aforementioned steps C1-C3 until the K1 identification result no longer changes. Assuming that the object re-identification within a single segment of K1 fails to update the K1 identification result during the candidate pool mechanism, the next candidate region (K1_A4) to match the K2 identification result is determined from K1_candidate_person_list, and this becomes the current region to be processed. K1_A4 is then matched against each matching region in K2_A_success_list and K2_B_success_list to obtain the similarity between K1_A4 and each matching region. The matching region with the highest similarity to K1_A4 (let's say K2_A9) is then determined. If the similarity between K1_A4 and K2_A9 is greater than a similarity threshold, K1_A4 is removed from K1_candidate_person_list and added to K1_A_success_list. K1_A_success_list is then updated, triggering object re-identification within a single segment of K1 based on the candidate pool mechanism to update the K1 identification result.This process is repeated until there are no matching regions in K1_candidate_person_list that are not matched against K2_A_success_list and K2_B_success_list. The process of correcting K1 using K2 is then complete.

[0164] When the electronic device acquires video K3, the processes of acquiring the recognition results of K1 and K2 based on the template image, as well as the bidirectional error correction between K1 and K2, have been completed. K1 and K2 are the second video, and K3 is the first video. The electronic device performs object re-identification within a single segment of K3 based on the candidate pool mechanism to obtain the K3 recognition results, including: K3_A_success_list=[K3_A2,K3_A4,K3_A6,K3_A8]; K3_B_success_list=[K3_B2,K3_B4,K3_B6,K3_B8]; K3_candidate_person_list=[K3_A1,K3_A3,K3_A5,K3_A7,K3_A9,K3_A10,K3_B1,K3_B3,K3_B5,K3_B7,K3_B9,K3_B10]. At this point, the electronic device has obtained the recognition results of K1 and K2, and both K3 and K2 are videos captured within the preset duration range of K1. Error correction between segments of K3, K1, and K2 is triggered. The electronic device can first use K1 to correct K3, that is, match the candidate regions added to K3_candidate_person_list with the matching regions in K1_A_success_list and K1_B_success_list. The method of using K1 to correct K3 is similar to the aforementioned method of using K1 to correct K2, and can be referred to the relevant description in the previous embodiment. After there are no matching regions in K3_candidate_person_list that are not matched with K1_A_success_list and K1_B_success_list, the process of using K1 to correct K3 is completed. K3 is used to correct K1 errors, which involves matching the candidate regions added to K1_candidate_person_list with the matching regions in K3_A_success_list and K3_B_success_list. The method of using K3 to correct K1 errors is similar to the method described above for using K2 to correct K1 errors; please refer to the relevant descriptions in the preceding embodiments. Once there are no matching regions in K1_candidate_person_list that do not match K3_A_success_list and K3_B_success_list, the process of using K3 to correct K1 errors is complete. During the cross-segment error correction process of K1 and K3, if the recognition result of any video changes, the video with the changed recognition result can be used to re-perform cross-segment error correction with K2.

[0165] Suppose that during the cross-segment error correction process for K1 and K3, the recognition result of K1 changes while the recognition result of K3 remains unchanged. After the cross-segment error correction for K1 and K3 is completed, the electronic device will re-perform cross-segment error correction for K1 and K2. If the recognition result of any video changes during the cross-segment error correction process for K1 and K2, the video with the changed recognition result can be used to re-perform cross-segment error correction with K3. This process is repeated until the recognition results of K1, K2, and K3 no longer change.

[0166] When the electronic device acquires video K4, the processes of acquiring the recognition results of K1, K2, and K3 based on the template image, as well as the error correction between K1, K2, and K3, have all been completed. K1, K2, and K3 are the second video, and K4 is the first video. Object re-identification is performed within a single segment of K4 based on a candidate pool mechanism to obtain the K4 recognition result. This process is repeated until the recognition results of K1, K2, K3, K4, and K5 no longer change. Object re-identification for K1 is then complete, and the resulting K1 recognition result is the final object recognition result for K1.

[0167] Similarly, electronic devices can correct the K5 recognition result based on the recognition results of K1-K4, correct the K6 recognition result based on the recognition results of K2-K5, and so on. Therefore, assuming the same object appears continuously in multiple videos, mutual correction using the recognition results of each video obtained within a short period can improve the accuracy of the obtained object recognition results. Furthermore, in practical scenarios, when calculating the similarity score between two images based on the distance between feature vectors to determine whether the two images match, for each object image region, only the image features of that object image region need to be extracted once. Subsequent calculations can directly calculate the distance between features based on the extracted image features, eliminating the need for repeated image feature extraction. This reduces the time consumed in the object re-recognition process and the computational resources required, improving the performance of the method. The extracted image features can be stored in a database.

[0168] In some embodiments, see Figure 2 , Figure 2 A flowchart illustrating the object re-identification result within a single segment obtained based on a candidate pool mechanism, provided in this application embodiment, is included. The method may include the following steps:

[0169] S201: Input fragment sequence frame.

[0170] In this step, the segment sequence frame refers to the first video captured by the aforementioned electronic device from the camera.

[0171] S202: Traverse the input fragment sequence frames.

[0172] In this step, the electronic device traverses the input segment sequence frames, namely the video frames to be identified in the first video as described above.

[0173] S203: Traverse the human-shaped bounding boxes within the sequence frames.

[0174] In this step, the sequence frame refers to the aforementioned video frame to be identified; the human-shaped frame refers to the aforementioned object image region. That is, the electronic device determines the object image region occupied by the object in the video frame to be identified.

[0175] S204: Match the human-shaped bounding box of the current frame with the registered template library sample.

[0176] In this step, the registered template library sample is the template image of the specified object in the aforementioned embodiment; the human-shaped bounding box of the current frame is the current area to be matched.

[0177] S205: Determine if the match was successful.

[0178] In this step, if the judgment result is yes, it means that the matching is successful, that is, the current area to be matched matches the template image, and step S206 is executed; if the judgment result is no, it means that the matching is unsuccessful, that is, the current area to be matched does not match the template image, and step S213 is executed.

[0179] S206: Add to the list of successfully identified samples.

[0180] In this step, the list of successfully identified samples is the aforementioned first matching region set; adding to the list of successfully identified samples means adding the current region to be matched to the first matching region set.

[0181] S207: Determine if there are any samples in the candidate pool sample list.

[0182] In this step, the candidate pool sample list is the aforementioned first candidate region set. If the judgment result is yes, it means that there are samples in the candidate pool sample list, and step S208 is executed; if the judgment result is no, it means that the candidate pool sample list is empty, and step S203 is executed.

[0183] S208: Iterate through the samples in the candidate pool sample list.

[0184] In this step, the electronic device traverses the samples in the candidate pool sample list, that is, the current region to be corrected is determined from the first candidate region set.

[0185] S209: Match the samples in the current candidate pool with the successfully identified samples.

[0186] In this step, the samples in the current candidate pool sample list are the aforementioned current regions to be corrected; the samples in the current candidate pool sample list are matched with the successfully identified samples, that is, the current regions to be corrected are matched with the current regions to be matched. Based on the matching results, the list of successfully identified samples can be updated, that is, step S206 is executed, adding the current regions to be corrected that match the current regions to be matched to the list of successfully identified samples.

[0187] S210: Determine if the match was successful.

[0188] In this step, if the judgment result is yes, it means that the matching is successful, that is, the current area to be corrected matches the current area to be matched, and step S211 is executed; if the judgment result is no, it means that the matching is unsuccessful, that is, the current area to be corrected does not match the current area to be matched, and step S208 is executed.

[0189] S211: Remove the sample from the candidate pool sample list and add it to the list of successfully identified samples.

[0190] In this step, the sample is removed from the candidate pool sample list, that is, the current region to be corrected in the first candidate region set is deleted; the sample is added to the list of successfully identified samples, that is, the current region to be corrected is added to the first matching region set.

[0191] S212: Determine whether the candidate pool sample list and the successfully identified sample list have changed.

[0192] In this step, if the judgment result is negative, it means that the candidate pool sample list and the successfully identified sample list have not changed, and step S214 is executed; if the judgment result is positive, it means that the candidate pool sample list and the successfully identified sample list have changed, and step S207 is executed.

[0193] S213: Add to the candidate pool sample list.

[0194] In this step, the sample is added to the candidate pool, which means adding the current region to be matched to the first candidate region set. The samples in the candidate pool sample list in step S208 are those added to the candidate pool sample list in step S213. Then, the electronic device returns to step S203 to continue traversing the human-shaped bounding boxes within the sequence frames.

[0195] In this embodiment of the application, the processing mechanism based on steps S205-S213 can be called the sample candidate pool mechanism (i.e., the aforementioned candidate pool mechanism).

[0196] S214: Determine whether the sequence frame traversal has ended.

[0197] In this step, if the judgment result is yes, it means that the sequence frame traversal has ended and the matching of each object image region in the first video has been performed, and step S215 is executed; if the judgment result is no, it means that the sequence frame traversal has not ended, and step S202 is executed.

[0198] S215: Output fragment matching results, including a list of successfully identified samples and a list of candidate pool samples.

[0199] In this step, the fragment matching result is the aforementioned object re-identification result, which includes the aforementioned first matching region set and first candidate region set.

[0200] In some embodiments, see Figure 3 , Figure 3 The flowchart for object re-identification with an error correction mechanism between fragment samples provided in this application embodiment includes the following steps:

[0201] S301: Input the recognition result of the current segment.

[0202] In this step, the current segment identification result is the aforementioned first matching region set and first candidate region set.

[0203] S302: Determine if there are any historical fragments within the validity period.

[0204] In this step, the validity period is the preset duration range prior to the first video; the historical segment is the second video. If the judgment result is yes, it means that the second video exists, and step S303 is executed; if the judgment result is no, it means that the second video does not exist, and step S312 is executed.

[0205] S303: Traverse historical fragments within the validity period.

[0206] In this step, the electronic device determines the second video currently used to correct the first video from the second video.

[0207] S304: Positive error correction: Traverse the candidate list of current segment samples and match it with the successful list of historical segment samples for error correction.

[0208] In this step, the current segment sample candidate list is the aforementioned first candidate region set; the historical segment sample successful list is the aforementioned second matching region set. In other words, the electronic device uses the second video to correct errors in the first video.

[0209] S305: Determine whether the sample in the current fragment sample candidate list has been successfully identified.

[0210] In this step, if the judgment result is yes, it means that the first matching region set has been updated using the second matching region set, and step S306 is executed; if the judgment result is no, it means that the first matching region set has not been updated, and step S307 is executed.

[0211] S306: Triggers the sample candidate pool mechanism for the current fragment sample.

[0212] In this step, the current segment sample's candidate pool mechanism, namely the aforementioned triggering of object re-identification within a single segment of the first video based on the candidate pool mechanism.

[0213] S307: Reverse error correction: Traverse the candidate list of historical fragment samples and match it with the successful list of current fragment samples for error correction.

[0214] In this step, the electronic device uses the first video to correct errors in the second video.

[0215] S308: Determine whether a sample in the historical fragment sample candidate list has been successfully identified.

[0216] In this step, if the judgment result is yes, it means that the second matching region set has been updated using the first matching region set, and step S309 is executed; if the judgment result is no, it means that the second matching region set has not been updated, and step S310 is executed.

[0217] S309: Triggering the sample candidate pool mechanism for historical fragment samples.

[0218] In this step, the historical segment sample candidate pool mechanism is used to trigger object re-identification within a single segment of the second video based on the aforementioned candidate pool mechanism.

[0219] S310: Determine whether either the historical fragment or the current fragment list has changed.

[0220] In this step, if the judgment result is yes, it means that the list of historical segments and the current segment has been updated, that is, the object recognition result of the first video and the object recognition result of the second video have changed, and step S304 is executed; if the judgment result is no, step S311 is executed.

[0221] S311: Determine if the traversal of the historical fragment has ended.

[0222] In this step, if the judgment result is yes, it means that the inter-segment error correction has been performed based on each second video and the first video, and step S313 is executed; if the judgment result is no, it means that the inter-segment error correction has not been completed, and step S303 is executed.

[0223] S312: Output the recognition result of the current segment.

[0224] In this step, the current segment recognition result is the object re-recognition result obtained only based on intra-segment matching.

[0225] S313: Output the matching results of the current segment and the historical segments after error correction.

[0226] In this step, the matching result between the current segment and the historical segments after error correction is the object re-identification result obtained after inter-segment error correction.

[0227] In the technical solution of this application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0228] Based on the same inventive concept as the object re-identification method described above, this application also provides an object re-identification device. See [link to relevant documentation]. Figure 4 , Figure 4 A structural diagram of an object re-identification device provided in an embodiment of this application is shown. The device includes:

[0229] The acquisition module 401 is used to acquire the video frame to be re-identified in the first video as the video frame to be identified;

[0230] The region determination module 402 is used to determine, from the object image region occupied by the object in the video frame to be identified, the next object image region to be matched with the template image of the specified object, as the current region to be matched;

[0231] The first matching module 403 is used to add the current region to be matched to the first candidate region set if the current region to be matched does not match the template image, and return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0232] The second matching module 404 is used to add the current region to be matched to the first matching region set of the specified object if the current region to be matched matches the template image, and add the first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set, and return to the step of determining the next object image region to be matched with the template image of the specified object from the object image region occupied by the object in the video frame to be identified.

[0233] Optionally, the device further includes: an inter-segment matching module, configured to, after completing the matching of all object image regions in the video frame to be identified of the first video, obtain a second matching region set of the second video; for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, add the first candidate region to the first matching region set.

[0234] Optionally, the apparatus further includes: a third matching module, configured to, before the inter-segment matching module performs the following steps for each first candidate region in the first candidate region set: if a second matching region exists in the second matching region set that matches the first candidate region, and before adding the first candidate region to the first matching region set; perform the following steps for each second candidate region in the second candidate region set: if a first matching region exists in the first matching region set that matches the second candidate region, and before adding the second candidate region to the second matching region set; or, a fourth matching module, configured to, after the inter-segment matching module performs the following steps for each first candidate region in the first candidate region set: if a second matching region exists in the second matching region set that matches the first candidate region, and before adding the first candidate region to the first matching region set; perform the following steps for each second candidate region in the second candidate region set: if a first matching region exists in the first matching region set that matches the second candidate region, and before adding the second candidate region to the second matching region set.

[0235] Optionally, the inter-segment matching module is specifically used for: determining the next first candidate region to be matched with the second matching region set from the first candidate region set, as the current region to be processed; if none of the second matching regions in the second matching region set match the current region to be processed, returning to the step of determining the next first candidate region to be matched with the second matching region set from the first candidate region set; if the current region to be processed matches any second matching region in the second matching region set, adding the current region to be processed to the first matching region set, and determining the current region to be processed as the new current region to be matched; adding the first candidate regions in the first candidate region set that match the current region to be matched to the first matching region set to the first matching region set; and returning to the step of determining the next first candidate region to be matched with the second matching region set from the first candidate region set.

[0236] Optionally, the first video is the most recently captured video, and the second video includes videos captured within a preset duration range prior to the first video.

[0237] Optionally, the step of adding the first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set includes: determining the next first candidate region to be matched with the current region to be matched from the first candidate region set, as the current region to be corrected; if the current region to be corrected does not match the current region to be matched, returning to the step of determining the next first candidate region to be matched with the current region to be matched from the first candidate region set; if the current region to be corrected matches the current region to be matched, adding the current region to be corrected to the first matching region set, and determining the current region to be corrected as the new current region to be matched; returning to the step of determining the next first candidate region to be matched with the current region to be matched from the first candidate region set; using the first matching region previously added to the first matching region set as the new current region to be matched; returning to the step of determining the next first candidate region to be matched with the current region to be matched from the first candidate region set.

[0238] Optionally, there may be multiple specified objects; a matching region in the matching region set of a specified object matches the template image of the specified object, or a matching region in the matching region set of the specified object matches any other matching region in the matching region set of the specified object.

[0239] Optionally, the acquisition module 401 is specifically used for: acquiring a first video; and extracting multiple video frames from the first video based on the shape of the object as video frames to be identified.

[0240] Optionally, the acquisition module 401 is specifically configured to: acquire preset keywords for each morphological dimension of the object to describe different forms of the object in that morphological dimension; perform image-text matching between each video frame in the first video and each preset keyword to obtain the matching degree between the video frame and each preset keyword; take the preset keywords whose matching degree with the video frame is not less than the matching degree threshold as the keywords to be extracted for the corresponding video frame; if the number of determined keywords to be extracted is not less than a preset number, for each keyword to be extracted, select one video frame from the video frames corresponding to the keyword to be extracted as the representative frame of the keyword to be extracted; select the preset number of representative frames from the representative frames of each keyword to be extracted as the video frames to be identified extracted from the first video; if the number of determined keywords to be extracted is less than the preset number, for each keyword to be extracted, select at least one video frame from the video frames corresponding to the keyword to be extracted to obtain the preset number of video frames as the video frames to be identified extracted from the first video.

[0241] This application also provides an object re-identification system, see [link to relevant documentation] Figure 5, Figure 5 This is an architecture diagram of an object re-identification system provided in an embodiment of this application. The system includes:

[0242] At least one first acquisition device 501 is used to acquire a first video frame containing an object image region;

[0243] At least one second acquisition device 502 is used to acquire a second video frame containing an object image region;

[0244] The processing device 503 is used to receive the first video frame acquired by the first acquisition device and the second video frame acquired by the second acquisition device, respectively, and to implement the object re-identification method described in any of the foregoing embodiments.

[0245] In the object re-identification system provided in this application, the first acquisition device 501 can be an indoor camera installed indoors or an outdoor camera installed outdoors, the second acquisition device 502 can be another indoor camera installed indoors or another outdoor camera installed outdoors, and the processing device 503 can be the aforementioned electronic device, which can be a local smart host or a remote cloud server. The first acquisition device 501 and the second acquisition device 502 have different perspectives, but the target object may appear in a certain area of ​​the captured image frame; that is, both can capture video frames containing the target object. The target object can be a person, vehicle, or animal. Figure 5 This description uses an example where the system includes one first acquisition device 501 and one second acquisition device 502. In actual scenarios, the number of first acquisition devices 501 and second acquisition devices 502 is not limited to this.

[0246] After receiving a video segment composed of the first video frames acquired by the first acquisition device 501, the processing device 503 can obtain an object re-identification result within a single video segment (hereinafter referred to as the first result) based on the aforementioned candidate pool mechanism. After receiving a video segment composed of the second video frames acquired by the second acquisition device 502, the processing device 503 can obtain an object re-identification result within a single video segment (hereinafter referred to as the second result) based on the aforementioned candidate pool mechanism. Furthermore, error correction between the aforementioned video segments can be performed based on the first result and the second result. During the error correction process between video segments, once the object re-identification result of any video segment is updated, object re-identification within that video segment can be triggered based on the aforementioned candidate pool mechanism.

[0247] Based on the object re-identification system provided in this application, object re-identification is performed within each video segment using a candidate pool mechanism. By matching the regions that match the template image again with the candidate regions, the candidate regions are further filtered, identifying any missed matching regions. Furthermore, object re-identification within the video segment can be triggered during the updating of the matching region set, also based on the candidate pool mechanism. This ensures that each candidate region is matched with as many matching regions as possible, reducing the probability of missed detections and improving the accuracy of the re-identification results.

[0248] Furthermore, after obtaining the object re-identification result within a single video segment based on the candidate pool mechanism, error correction between video segments can be performed based on the object re-identification results of other video segments, so that each candidate region can be matched with as many matching regions as possible, which can further improve the accuracy of the obtained re-identification results.

[0249] This application also provides an electronic device, such as... Figure 6 As shown, it includes:

[0250] Memory 601 is used to store computer programs;

[0251] When the processor 602 executes the program stored in the memory 601, it implements any of the object re-identification methods described in the foregoing embodiments.

[0252] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 602, communication interface, and memory 601 communicating with each other via the communication bus.

[0253] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0254] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0255] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0256] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0257] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described object re-identification methods.

[0258] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the object re-identification methods described above.

[0259] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0260] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0261] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, systems, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0262] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for object re-identification, characterized in that, The method comprises: acquiring a video frame to be re-identified in a first video as a to-be-identified video frame; determining, from an object image region occupied by an object in the to-be-identified video frame, a next object image region to be matched with a template image of a specified object as a current to-be-matched region; if the current to-be-matched region does not match the template image, adding the current to-be-matched region to a first candidate region set, and returning to the step of determining, from an object image region occupied by an object in the to-be-identified video frame, a next object image region to be matched with a template image of a specified object; if the current to-be-matched region matches the template image, adding the current to-be-matched region to a first matching region set of the specified object, and adding a first candidate region in the first candidate region set that matches the current to-be-matched region to the first matching region set, and returning to the step of determining, from an object image region occupied by an object in the to-be-identified video frame, a next object image region to be matched with a template image of a specified object.

2. The method of claim 1, wherein, After completing the matching of all object image regions in the to-be-identified video frame of the first video, the method further comprises: acquiring a second matching region set of a second video; for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set.

3. The method of claim 2, wherein, Before the step of for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set, the method further comprises: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, adding the second candidate region to the second matching region set; or, After the step of for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set, the method further comprises: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, adding the second candidate region to the second matching region set.

4. The method of claim 2, wherein, The step of for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set, comprises: determining, from the first candidate region set, a next first candidate region to be matched with the second matching region set as a current to-be-processed region; if none of the second matching regions in the second matching region set matches the current to-be-processed region, returning to the step of determining, from the first candidate region set, a next first candidate region to be matched with the second matching region set; If the current to-be-processed region matches any second matching region in the second matching region set, the current to-be-processed region is added to the first matching region set, and the current to-be-processed region is determined as a new current to-be-matched region; a first candidate region in the first candidate region set that matches the current to-be-matched region is added to the first matching region set; and the step of determining, from the first candidate region set, a next first candidate region that matches the current to-be-matched region in the second matching region set is performed.

5. The method according to any one of claims 2-4, characterized in that, The first video is a newly collected video, and the second video includes a video collected within a preset time range before the first video.

6. The method according to any one of claims 1 to 4, characterized in that, The adding of the first candidate region in the first candidate region set that matches the current to-be-matched region to the first matching region set includes: determining, from the first candidate region set, a next first candidate region that matches the current to-be-matched region as a current to-be-error-corrected region; if the current to-be-error-corrected region does not match the current to-be-matched region, performing the step of determining, from the first candidate region set, a next first candidate region that matches the current to-be-matched region; if the current to-be-error-corrected region matches the current to-be-matched region, adding the current to-be-error-corrected region to the first matching region set, and determining the current to-be-error-corrected region as a new current to-be-matched region; and performing the step of determining, from the first candidate region set, a next first candidate region that matches the current to-be-matched region; taking a first matching region that is added to the first matching region set in the last time as a new current to-be-matched region; and performing the step of determining, from the first candidate region set, a next first candidate region that matches the current to-be-matched region.

7. The method according to any one of claims 1 to 4, characterized in that, The specified object is a plurality of objects; a matching region in a matching region set of one specified object matches a template image of the specified object, or a matching region in the matching region set of the specified object matches any other matching region in the matching region set of the specified object.

8. The method according to any one of claims 1 to 4, characterized in that, The obtaining of the video frame to be re-identified in the first video as the to-be-identified video frame includes: obtaining the first video; extracting a plurality of video frames from the first video as the to-be-identified video frames based on the morphologies of the object.

9. The method of claim 8, wherein, The extracting of the plurality of video frames from the first video as the to-be-identified video frames based on the morphologies of the object includes: for each morphology dimension of the object, obtaining each preset keyword used to describe different morphologies of the object in the morphology dimension; for each video frame in the first video, performing image-text matching on the video frame and each preset keyword to obtain a matching degree of the video frame and each preset keyword; taking a preset keyword with a matching degree of the video frame being not less than a matching degree threshold as a to-be-framed keyword corresponding to the video frame; if the number of the determined to-be-framed keywords is not less than a preset number, for each to-be-framed keyword, selecting a video frame from the video frame corresponding to the to-be-framed keyword as a representative frame of the to-be-framed keyword; and selecting the preset number of representative frames from the representative frames of the to-be-framed keywords as the to-be-identified video frames extracted from the first video. If the number of the determined frame-to-be-extracted keywords is less than the preset number, for each frame-to-be-extracted keyword, at least one video frame is selected from the video frame corresponding to the frame-to-be-extracted keyword, and the preset number of video frames are obtained as the frame-to-be-identified video frames extracted from the first video.

10. An object re-identification apparatus characterized by comprising: The apparatus comprises: an acquisition module configured to acquire a video frame to-be-reidentified in a first video as a frame-to-be-identified; a region determination module configured to determine, from an object image region occupied by an object in the frame-to-be-identified, a next object image region to be matched with a template image of a specified object as a current region to be matched; a first matching module configured to, if the current region to be matched does not match the template image, add the current region to be matched to a first candidate region set, and return to the step of determining, from an object image region occupied by an object in the frame-to-be-identified, a next object image region to be matched with the template image of the specified object; a second matching module configured to, if the current region to be matched matches the template image, add the current region to be matched to a first matching region set of the specified object, add a first candidate region in the first candidate region set that matches the current region to be matched to the first matching region set, and return to the step of determining, from an object image region occupied by an object in the frame-to-be-identified, a next object image region to be matched with the template image of the specified object.

11. The apparatus of claim 10, wherein, The apparatus further comprises: an inter-segment matching module configured to, after completing matching of all object image regions in the frame-to-be-identified of the first video, perform the following steps: acquiring a second matching region set of a second video; and for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set; and / or The apparatus further comprises: a third matching module configured to, before the inter-segment matching module performs the following step: for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set, perform the following step: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, adding the second candidate region to the second matching region set; or a fourth matching module configured to, after the inter-segment matching module performs the following step: for each first candidate region in the first candidate region set, if there is a second matching region in the second matching region set that matches the first candidate region, adding the first candidate region to the first matching region set, perform the following step: for each second candidate region in the second candidate region set, if there is a first matching region in the first matching region set that matches the second candidate region, adding the second candidate region to the second matching region set; and / or The inter-segment matching module is specifically configured to: determine, from the first candidate region set, a next first candidate region to be matched with the second matching region set as a current region to be processed; If none of the second matching regions in the second matching region set matches the current to-be-processed region, the step of determining, from the first candidate region set, a next first candidate region to be matched with the second matching region set is performed again. If the current to-be-processed region matches any second matching region in the second matching region set, the current to-be-processed region is added to the first matching region set, and the current to-be-processed region is determined as a new current to-be-processed region; a first candidate region in the first candidate region set that matches the current to-be-processed region is added to the first matching region set; and the step of determining, from the first candidate region set, a next first candidate region to be matched with the current to-be-processed region is performed again. And / or, The first video is a newly collected video, and the second video includes a video collected within a preset time range before the first video. And / or, The step of adding, to the first matching region set, a first candidate region in the first candidate region set that matches the current to-be-processed region includes: Determining, from the first candidate region set, a next first candidate region to be matched with the current to-be-processed region as a current to-be-error-corrected region; If the current to-be-error-corrected region does not match the current to-be-processed region, the step of determining, from the first candidate region set, a next first candidate region to be matched with the current to-be-processed region is performed again. If the current to-be-error-corrected region matches the current to-be-processed region, the current to-be-error-corrected region is added to the first matching region set, and the current to-be-error-corrected region is determined as a new current to-be-processed region; and the step of determining, from the first candidate region set, a next first candidate region to be matched with the current to-be-processed region is performed again. A first matching region that was added to the first matching region set last time is taken as a new current to-be-processed region; and the step of determining, from the first candidate region set, a next first candidate region to be matched with the current to-be-processed region is performed again. And / or, The specified object is a plurality of objects; a matching region in a matching region set of one specified object matches a template image of the specified object, or a matching region in the matching region set of the specified object matches any other matching region in the matching region set of the specified object; And / or, The obtaining module is specifically configured to: Obtain the first video; Extract a plurality of video frames from the first video as to-be-recognized video frames based on the morphologies of the object; And / or, The obtaining module is specifically configured to: For each morphology dimension of the object, obtain each preset keyword used to describe different morphologies of the object in the morphology dimension; For each video frame in the first video, perform image-text matching on the video frame and each preset keyword to obtain a matching degree of the video frame and each preset keyword; A preset keyword with a matching degree not less than a matching degree threshold of the video frame is taken as a to-be-framed keyword corresponding to the video frame. If the number of the determined frame-to-be-extracted keywords is not less than the preset number, for each frame-to-be-extracted keyword, a video frame corresponding to the frame-to-be-extracted keyword is selected as a representative frame of the frame-to-be-extracted keyword; and the preset number of representative frames are selected from the representative frames of the frame-to-be-extracted keywords as the extracted video frames to be recognized from the first video. If the number of the determined frame-to-be-extracted keywords is less than the preset number, for each frame-to-be-extracted keyword, at least one video frame corresponding to the frame-to-be-extracted keyword is selected to obtain the preset number of video frames as the extracted video frames to be recognized from the first video.

12. An object re-identification system, comprising: The system comprises: at least one first acquisition device configured to acquire a first video frame containing an object image region; at least one second acquisition device configured to acquire a second video frame containing an object image region; a processing device configured to receive the first video frame acquired by the first acquisition device and the second video frame acquired by the second acquisition device, and implement the method of any one of claims 1-9.

13. An electronic device, comprising: comprise: a memory configured to store a computer program; a processor configured to execute the program stored in the memory, and implement the method of any one of claims 1-9.

14. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to implement the method of any one of claims 1-9.

15. A computer program product, characterised in that, When the computer program product runs on the computer, the computer is caused to execute the method of any one of claims 1-9.

Images