Matching method and apparatus, storage medium and electronic device

Abstract

The application provides a matching method and device, a storage medium and an electronic device. The method comprises the following steps: determining a to-be-recognized detection frame corresponding to a to-be-recognized image and a target detection frame, wherein the to-be-recognized detection frame contains a detected to-be-recognized object, and the target detection frame is determined based on a target object; in the case that it is determined that the to-be-recognized detection frame and the target detection frame are not matched, determining a to-be-matched feature of the to-be-recognized object in the to-be-recognized image; and determining a matching degree of the to-be-recognized object and the target object based on the to-be-matched feature. By using the technical solution, the problem that the success rate of object matching is low due to the fact that the target is easily blocked in the related art is solved.

Description

[Technical Field]

[0001] This application relates to the field of communications, and more specifically, to a matching method and apparatus, a storage medium, and an electronic device. [Background Technology]

[0002] Currently, target following can be achieved through machine object matching, where the target can be the person the machine wants to follow. When the machine follows a person's movement, it needs to accurately locate and identify the target, that is, match the target. In real-world scenarios, occlusion can occur, causing the target to be lost, thus preventing the machine from effectively following.

[0003] This shows that the object matching methods in related technologies have a low matching success rate. [Summary of the Invention]

[0004] The purpose of this application is to provide a matching method and apparatus, storage medium and electronic device, so as to at least solve the problem that the object matching method in the related art has a low success rate due to the target being easily occluded.

[0005] The purpose of this application is to achieve the following technical solution:

[0006] According to one aspect of the embodiments of this application, a matching method is provided, the method comprising: determining a detection box to be identified and a target detection box corresponding to an image to be identified, wherein the detection box to be identified contains a detected object to be identified, and the target detection box is determined based on the target object; when it is determined that the detection box to be identified and the target detection box do not match, determining a matching feature of the object to be identified in the image to be identified; and determining the matching degree between the object to be identified and the target object based on the matching feature.

[0007] In an exemplary embodiment, determining that the detection box to be identified does not match the target detection box includes: determining the target overlap between the detection box to be identified and the target detection box; and determining that the detection box to be identified does not match the target detection box if the target overlap is less than or equal to a first preset threshold.

[0008] In an exemplary embodiment, determining that the detection box to be identified does not match the target detection box includes: determining that the detection box to be identified does not match the target detection box when the number of times the target detection box has been used for object matching is greater than a second preset threshold.

[0009] In an exemplary embodiment, determining the matching degree between the object to be identified and the target object based on the matching feature includes: determining the feature similarity between the matching feature and the target object feature of the target object; determining that the object to be identified matches the target object if the feature similarity is greater than or equal to a third preset threshold; and determining that the object to be identified matches the target object if the feature similarity is less than the third preset threshold.

[0010] In an exemplary embodiment, after determining the detection box to be identified and the target detection box corresponding to the image to be identified, the method further includes: if it is determined that the object to be identified matches the target object, updating the target detection box using the detection box to be identified to obtain the updated target detection box.

[0011] In an exemplary embodiment, after updating the target detection box using the detection box to be identified to obtain the updated target detection box, the method further includes: using a depth image matching the image to be identified to determine the position information of the target object in the target detection box; and controlling a following device of the target object to follow the target object to move according to the position information of the target object.

[0012] In an exemplary embodiment, before determining the position information of the target object in the target detection box using a depth image matched with the image to be identified, the method further includes: performing position matching between the image to be identified and the depth image to obtain position mapping information between the image to be identified and the depth image, wherein the position mapping information is used to perform position mapping between the image to be identified and the depth image.

[0013] According to another aspect of the embodiments of this application, a matching device is also provided, the device comprising: a first determining unit, configured to determine a detection box to be identified and a target detection box corresponding to an image to be identified, wherein the detection box to be identified contains a detected object to be identified, and the target detection box is determined based on the target object; a second determining unit, configured to determine a matching feature of the object to be identified in the image to be identified when it is determined that the detection box to be identified and the target detection box do not match; and a third determining unit, configured to determine the matching degree between the object to be identified and the target object based on the matching feature.

[0014] In one exemplary embodiment, the apparatus further includes: a fourth determining unit, configured to determine the target overlap between the detection box to be identified and the target detection box; and a fifth determining unit, configured to determine that the detection box to be identified and the target detection box do not match if the target overlap is less than or equal to a first preset threshold.

[0015] In one exemplary embodiment, the apparatus further includes a sixth determining unit, configured to determine that the detection box to be identified does not match the target detection box when the number of times the target detection box has been used for object matching is greater than a second preset threshold.

[0016] In an exemplary embodiment, the third determining unit includes: a first determining module, configured to determine the feature similarity between the feature to be matched and the target object feature of the target object; a second determining module, configured to determine that the object to be identified matches the target object when the feature similarity is greater than or equal to a third preset threshold; and a third determining module, configured to determine that the object to be identified matches the target object when the feature similarity is less than the third preset threshold.

[0017] In one exemplary embodiment, the apparatus further includes: an update unit, configured to, after determining the detection box to be identified and the target detection box corresponding to the image to be identified, update the target detection box using the detection box to be identified, and obtain the updated target detection box if it is determined that the object to be identified matches the target object.

[0018] In one exemplary embodiment, the apparatus further includes: a seventh determining unit, configured to, after updating the target detection box using the detection box to be identified to obtain the updated target detection box, determine the position information of the target object in the target detection box using a depth image matching the image to be identified; and a control unit, configured to control a following device of the target object to follow the target object and move according to the position information of the target object.

[0019] In one exemplary embodiment, the apparatus further includes: a matching unit, configured to perform position matching between the image to be identified and the depth image before determining the position information of the target object in the target detection box using a depth image that matches the image to be identified, to obtain position mapping information between the image to be identified and the depth image, wherein the position mapping information is used to perform position mapping between the image to be identified and the depth image.

[0020] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the test method of the above-described interface at runtime.

[0021] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the test method of the interface described above through the computer program.

[0022] In this embodiment, a combination of detection box matching and feature matching is used. The method involves determining the detection box to be identified and the target detection box corresponding to the image to be identified. The detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object. If the detection box to be identified and the target detection box do not match, the matching features of the object to be identified in the image to be identified are determined. The matching degree between the object to be identified and the target object is determined based on the matching features. Because matching is first performed based on the detection box to be identified (the detection box to be identified in the current video frame) and the target detection box (the tracking box, i.e., the detection box identified in the previous image that matches the target object), identification is still possible even when the target is partially occluded. When the detection box matching fails, matching is then performed based on the object features, ensuring accurate identification even when the target reappears after being lost. This achieves the goal of effectively following the target (or tracking the target), improving the success rate of object matching and solving the problem of low success rate in related object matching methods due to target occlusion. [Attached Image Description]

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the hardware environment for an optional matching method according to an embodiment of this application;

[0026] Figure 2 This is a flowchart illustrating an optional matching method according to an embodiment of this application;

[0027] Figure 3This is a flowchart illustrating another optional matching method according to an embodiment of this application;

[0028] Figure 4 This is a structural block diagram of an optional matching device according to an embodiment of this application;

[0029] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application.

Detailed Implementation Methods

[0030] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0032] According to one aspect of the embodiments of this application, a matching method is provided. Optionally, in this embodiment, the above-described matching method can be applied to, for example... Figure 1 The hardware environment shown consists of a mobile robot 102 and a server 104. Figure 1 As shown, server 104 is connected to mobile robot 102 via a network and can be used to provide services (such as application services) to mobile robot or clients installed on mobile robot. A database can be set up on the server or independently of the server to provide data storage services for server 104.

[0033] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The mobile robot 102 may not be limited to: a follower robot, a multi-legged robot, etc.

[0034] The matching method of this application embodiment can be executed by server 104, mobile robot 102, or jointly by server 104 and mobile robot 102. Alternatively, the matching method of this application embodiment can be executed by a client installed on mobile robot 102.

[0035] Taking the matching method in this embodiment executed by mobile robot 102 as an example, Figure 2 This is a flowchart illustrating an optional matching method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include the following steps:

[0036] Step S202: Determine the detection box to be identified and the target detection box corresponding to the image to be identified. The detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object.

[0037] The matching method in this embodiment can be applied to scenarios where target following is performed through object matching. The target identified through object matching is then followed; this can be either an object matching method or a target matching method. The target to be identified can be the target that the machine wants to follow (e.g., a specific human body). The machine can be a mobile robot used for target following, or other machines with similar target recognition functions. This embodiment uses a mobile robot as an example for illustration.

[0038] A mobile robot can be equipped with image acquisition devices (e.g., cameras), and can be a mobile robot that moves to follow a specific target. During its movement, the mobile robot can acquire images captured by the image acquisition devices and perform target detection based on a target template. Target detection can be performed frame-by-frame; that is, the mobile robot sequentially identifies objects in each image captured by the image acquisition devices that match the target being followed.

[0039] For an image to be identified (which could be the current video frame), the mobile robot can perform object detection (e.g., human detection) to obtain a bounding box containing the detected object. Simultaneously, the mobile robot can also determine a target bounding box, which is determined based on the target object. This target bounding box can be a detection box identified in a preceding image (which could be a preceding video frame) that matches the target object. For the image to be identified, this target bounding box can be considered as the target object's tracking box (or follow box).

[0040] It should be noted that target object detection is performed for each frame of the image. If a target object is detected (which can match the target object template), the location of the target object can be indicated by a detection box. In other words, the detection box is a box (which can be a rectangle) used to mark the location of the target object in a frame of the image. At the same time, the detection box also serves as the tracking box for the next frame of the image. For the next frame of the image, object detection is also performed, and after detection, the detection box for that frame and the tracking box obtained from the previous frame are obtained.

[0041] Step S204: If it is determined that the detection box to be identified does not match the target detection box, determine the matching features of the object to be identified in the image to be identified.

[0042] The mobile robot can determine whether the detection box to be identified and the target detection box match based on their information. There are various ways to determine whether the detection box to be identified and the target detection box match, including but not limited to matching based on overlap area or based on intersection over union (IOU). This embodiment does not limit this approach.

[0043] If it is determined that the bounding box to be identified does not match the target bounding box, the mobile robot can perform object matching based on features. For example, the mobile robot can extract the matching features of the object to be identified from the image to be identified. For instance, it can first extract the target image from the image to be identified using the bounding box, and then perform feature extraction to obtain the matching features. The extracted features can include one or more, and can include, but are not limited to, at least one of the following: body shape features, clothing features, etc.

[0044] Step S206: Determine the matching degree between the object to be identified and the target object based on the features to be matched.

[0045] Based on the features to be matched, a mobile robot can determine the degree of matching between the object to be identified and the target object. For example, the mobile robot can first acquire the target object features. These target object features can be the object features extracted from the target template (i.e., the template of the target object). Based on the feature similarity between the features to be matched and the target object features, the degree of matching between the object to be identified and the target object can be determined. There is a positive correlation between feature similarity and object matching degree; the higher the feature similarity, the higher the object matching degree; conversely, the lower the feature similarity, the lower the object matching degree.

[0046] Optionally, the feature similarity between the feature to be matched and the target object feature can be measured by the cosine distance between them. The smaller the cosine distance, the higher the feature similarity; conversely, the larger the cosine distance, the lower the feature similarity. Besides cosine distance, other measures that can measure the magnitude of differences between object features can also be applied in this embodiment.

[0047] Through steps S202 to S206, the detection bounding box and the target detection box corresponding to the image to be identified are determined. The detection bounding box contains the detected object to be identified, and the target detection box is determined based on the target object. If the detection bounding box and the target detection box do not match, the matching features of the object to be identified in the image to be identified are determined. The matching degree between the object to be identified and the target object is determined based on the matching features. This solves the problem of low success rate of object matching due to the easy occlusion of the target in the object matching method of related technologies, and improves the success rate of object matching.

[0048] In one exemplary embodiment, determining that the detection box to be identified does not match the target detection box includes:

[0049] S11, Determine the target overlap between the detection box to be identified and the target detection box;

[0050] S12, if the target overlap is less than or equal to the first set threshold, determine that the detection box to be identified does not match the target detection box.

[0051] To effectively improve the real-time performance of object matching, the overlap between the detection box to be identified and the target detection box can be used to measure whether they match. The mobile robot can determine the overlap between the detection box to be identified and the target detection box based on their position information, thus obtaining the target overlap. If the target overlap is less than or equal to a first preset threshold (i.e., the preset overlap threshold), the mobile robot can determine that the detection box to be identified and the target detection box do not match.

[0052] If the overlap between the targets is greater than a first set threshold, the mobile robot can directly determine that the detection box to be identified matches the target detection box. Optionally, in order to improve the accuracy of object matching, the mobile robot can also determine whether the number of times the target detection box has been used for object matching exceeds a set threshold, i.e., a second set threshold (i.e., the number of times the tracking box has been used for matching is within the valid number). If so, it can be determined that the detection box to be identified does not match the target detection box.

[0053] In this embodiment, by matching the Intersection over Union (IOU) of the tracking box and the detection box, and performing detection box matching based on the IOU, the efficiency of object matching can be effectively improved, thereby improving the real-time performance of object tracking. In an exemplary embodiment, determining that the detection box to be identified does not match the target detection box includes:

[0054] S21, if the number of times the target detection box has been used for object matching is greater than the second set threshold, it is determined that the detection box to be identified does not match the target detection box.

[0055] In this embodiment, the mobile robot can determine whether the number of times the target detection box has been used for object matching exceeds a second preset threshold. If so, it can be directly determined that the detection box to be identified does not match the target detection box. Otherwise, detection box matching is performed based on IOU or other metrics. The method of detection box matching based on IOU is similar to that in the previous embodiments and will not be described in detail here.

[0056] This embodiment determines whether the object to be identified matches the target object based on the number of consecutive tracking box matching attempts within a valid range, which avoids information accumulation errors and improves the accuracy of object matching.

[0057] In an exemplary embodiment, determining the matching degree between the object to be identified and the target object based on the features to be matched includes:

[0058] S31, Determine the feature similarity between the feature to be matched and the target object feature of the target object;

[0059] S32, if the feature similarity is greater than or equal to the third set threshold, determine that the object to be identified matches the target object;

[0060] S33, if the feature similarity is less than the third set threshold, determine that the object to be identified matches the target object.

[0061] When determining the matching degree between the object to be identified and the target object, the mobile robot can first determine the feature similarity between the features to be matched and the features of the target object. This feature similarity can be cosine similarity. If the feature similarity is greater than or equal to a third preset threshold (i.e., the cosine distance between features is less than or equal to a preset distance threshold), the object to be identified can be determined to match the target object. Otherwise, the object to be identified can be determined to match the target object.

[0062] Optionally, in this embodiment, the image to be identified may contain multiple detection boxes to be identified, corresponding to the number of objects to be identified being multiple. The mobile robot can sequentially determine whether each detection box to be identified matches the target detection box. When the detection box to be identified with the highest degree of matching with the target detection box (e.g., the highest overlap) does not meet the matching condition with the target detection box (e.g., the overlap is higher than a first set threshold), feature extraction is performed, and the matching between the object to be identified and the target object is determined based on the object features.

[0063] Determining whether an object to be identified matches a target object based on object features can be performed on all objects to be identified, or it can be performed on objects in the detection box that match the target detection box with the highest degree of matching, or it can be performed on objects in the detection box that match the target detection box with a degree of matching higher than a set matching threshold. This embodiment does not limit this.

[0064] In this embodiment, the matching degree between the object to be identified and the target object is determined based on feature similarity and a set similarity threshold, which can improve the efficiency of object matching.

[0065] In one exemplary embodiment, after determining the detection box to be identified and the target detection box corresponding to the image to be identified, the above method further includes:

[0066] S41, if it is determined that the object to be identified matches the target object, the target detection box is updated using the detection box to be identified, and the updated target detection box is obtained.

[0067] If the object to be identified matches the target object based on bounding box matching or feature matching, then the object to be identified is the object in the image that matches the target object; that is, the matching object of the target object. The mobile robot can then proceed to the object matching process for the next frame.

[0068] To improve the accuracy of object matching, when it is determined that the object to be identified matches the target object, the detection box to be identified can be used to update the target detection box to obtain the updated target detection box.

[0069] The candidate object is used as a reference object when performing object recognition on video frames following the current video frame. In this case, there can be one or more reference objects. To reduce the storage resources required to store the object information (e.g., reference object features) of the reference object, the reference object can be updated using the candidate object to obtain the updated reference object. In this case, there can be only one reference object.

[0070] In this embodiment, updating the reference object using the matching object of the target object in the current video frame can improve the accuracy of object recognition while reducing the storage resources required to store the object information of the reference object.

[0071] In one exemplary embodiment, after updating the target detection box using the detection box to be identified to obtain the updated target detection box, the above method further includes:

[0072] S51, using a depth image matched with the image to be identified, determine the location information of the target object in the target detection box;

[0073] S52, control the following device of the target object to move according to the position information of the target object.

[0074] To perform effective target following, a mobile robot needs to determine the position of the target object. To improve tracking accuracy, the mobile robot can use a depth image matched with the image to be identified to determine the position information of the target object within the target detection box (which can be understood as the tracking box of the target object). Here, a depth image refers to an image that uses the distance (depth) values ​​from the image acquisition device to each point in the scene as pixel values. Optionally, the mobile robot can first determine the matching object of the target object in the depth image, and then use the configuration information of the matching object to determine the position information of the target object in the target detection box.

[0075] The target object is the target that the mobile robot follows. After determining the position information of the target object, the following device of the target object (i.e., the aforementioned target robot) can be controlled to follow the target object according to its position information.

[0076] For example, after each update of the tracking bounding box, the mobile robot can use the depth image to determine the coordinates of the target and control its own movement based on those coordinates. The tracking described above can be done at intervals, which can be dynamically adjusted. For example, the more objects in the field of view, the closer the interval (to avoid losing the target). In other words, the interval between the following device and the target object is dynamically adjusted according to the number of objects to be identified in the image; the more objects in the image, the closer the interval.

[0077] This embodiment uses depth images to determine the position of the target being followed, and controls the movement of the following device based on the determined position, which can improve the effectiveness of target following.

[0078] In an exemplary embodiment, before determining the location information of the target object within the target detection box using a depth image matched with the image to be identified, the method further includes:

[0079] S61, perform position matching between the image to be identified and the depth image to obtain position mapping information between the image to be identified and the depth image, wherein the position mapping information is used to perform position mapping between the image to be identified and the depth image.

[0080] The image to be identified and the depth image can be acquired by different image acquisition devices on the mobile robot, or they can be acquired by the same image acquisition device. If different image acquisition devices are used, the positions of the target object in the image to be identified and the depth image will not be completely consistent due to the influence of shooting angle, shooting parameters, and other factors.

[0081] To improve the accuracy of target object location determination, the mobile robot can first perform position matching between the image to be identified and the depth image, thereby determining the position mapping information between the two images. This position mapping information is used to map the position of the target object in the image to be identified to its position in the depth image. By using the position mapping information, the position information of the target object in the image to be identified can be mapped to its position information in the depth image.

[0082] This embodiment improves the accuracy of target object location determination by performing position matching between the image to be identified and the depth image, thereby enhancing the effectiveness of target tracking.

[0083] The matching method in this embodiment will be explained below with reference to an optional example. In this optional example, the mobile robot is an omnidirectional robot, the target object is a tracking target (i.e., a specific human body), the detection box to be identified is a candidate detection box (i.e., a human body box), the target detection box is a tracking box, and the image to be identified is an RGB image.

[0084] Template-based feature matching suffers from a difficulty in matching after scale changes, making effective tracking challenging. This optional example provides a human tracking solution for an omnidirectional robot that employs real-time detection and recognition. By detecting the target first and then extracting features, it avoids changes in feature scale, resulting in more accurate matching and achieving real-time, accurate human tracking.

[0085] like Figure 3 As shown, the matching method in this optional example may include the following steps:

[0086] Step S302, target detection.

[0087] Omnidirectional robots can acquire RGB images and depth images (corresponding to depth point clouds) from an image acquisition device and perform position matching on the images. The RGB images can be input into a matching module (which corresponds to a program capable of object matching). For the input RGB image, the matching module can use the RGB image to detect human bounding boxes. That is, it performs object detection (i.e., human detection) on the RGB image and obtains the object detection result. The object detection result can be the detection result of a human body.

[0088] Step S304, post-processing.

[0089] The matching module can perform post-processing on the target detection results, such as filtering, to facilitate subsequent operations.

[0090] Step S306: Determine if there is a target to be detected. If there is, proceed to step S308; otherwise, proceed to step S302.

[0091] The matching module can determine whether there is a detected target based on the target detection result (this can be determined by checking whether there is a detected human bounding box). If there is, proceed to step S308; otherwise, increment the number of matching failures (loss) by 1, return to step S302, and proceed to the next frame.

[0092] Step S308: Determine if there is a target to track. If there is, proceed to step S310; otherwise, proceed to step S320.

[0093] The matching module can match tracking boxes and detection boxes. For example, it can perform IOU matching between tracking boxes and detection boxes. If a match is found, step S310 is executed; otherwise, step S320 is executed.

[0094] Step S310: Delete invalid tracking targets.

[0095] When the target being tracked is occluded by other targets, mismatches may occur during IOU matching between the tracking bounding box and the detection bounding box. If not corrected in time, this could lead to the wrong direction. When there is only one person, there are no other targets, so the box matching is effective. However, when there are more than one person, if the number of consecutive box matching exceeds the maximum number of matches MAX_BOX_MATCH_TIMES (i.e., max_loss, which can be set to 10), feature matching needs to be called to ensure that the current target being tracked is effective.

[0096] The matching module can delete tracking boxes that have been used for matching more than the valid number of times, thus removing invalid tracking targets. The valid number of times can be the maximum number of times a tracking target is allowed to be lost (max_loss mentioned above), which is the maximum number of times a match fails.

[0097] Step S312: Determine if there is a target to track. If there is, proceed to step S302; otherwise, proceed to step S314.

[0098] If there is a target to track (i.e., the detection box matches the tracking box, and the number of consecutive tracking box matches is within the valid number), the match can be determined to be successful, the tracking box is updated (the loss of the updated tracking box is 0), and the process moves to the next frame.

[0099] Step S314, Feature matching.

[0100] The matching module can extract the target image from the detection box, perform feature extraction, and match the extracted features with the target template features (features in the tracking target template) of each tracked target.

[0101] Step S316: Determine if the match is successful. If yes, proceed to step S318; otherwise, proceed to step S302.

[0102] If the similarity between the target template features (features in the target template) and the extracted features meets the set threshold (the similarity reaches the set similarity threshold), the match is successful and step S318 is executed; otherwise, the match fails and step S302 is executed to proceed to the next frame.

[0103] Step S318, update the tracking box (the loss of the updated tracking box is 0).

[0104] Step S320: Select target.

[0105] Mobile robots can interact with users through terminals or other means to determine the target selected by the user.

[0106] Step S322, set the target.

[0107] The mobile robot can set a selected target as the tracking target, thereby obtaining a tracking target template. Optionally, the tracking target template can also be preset.

[0108] Upon entering the next frame, target detection is performed again. After detection, the detection bounding box of the current frame and the tracking bounding box obtained from the previous frame are obtained. Then, the Intersection over Union (IOU) is calculated to determine whether there is a match. If there is no match, features are extracted for comparison. The specific process is similar to that described above and will not be repeated here.

[0109] This example demonstrates how real-time detection and recognition can be used to detect the target first and then extract features, avoiding changes in feature scale and making the matching more accurate. This enables real-time and accurate human body tracking, improving the accuracy of the tracking.

[0110] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0112] According to another aspect of the embodiments of this application, a matching apparatus for implementing the above-described matching method is also provided. Figure 4 This is a structural block diagram of an optional matching device according to an embodiment of this application, such as... Figure 4 As shown, the device may include:

[0113] The first determining unit 402 is used to determine the detection box to be identified and the target detection box corresponding to the image to be identified, wherein the detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object;

[0114] The second determining unit 404 is connected to the first determining unit 402 and is used to determine the matching features of the object to be identified in the image when it is determined that the detection box to be identified does not match the target detection box.

[0115] The third determining unit 406, connected to the second determining unit 404, is used to determine the matching degree between the object to be identified and the target object based on the features to be matched.

[0116] It should be noted that the first determining unit 402 in this embodiment can be used to execute the above-described step S202, the second determining unit 404 in this embodiment can be used to execute the above-described step S204, and the third determining unit 406 in this embodiment can be used to execute the above-described step S206. The above-described matching device can be an object matching device or a target matching device.

[0117] The above modules determine the matching degree between the object to be identified and the target object based on the features to be matched.

[0118] Based on the features to be matched, a mobile robot can determine the degree of matching between the object to be identified and the target object. For example, the mobile robot can first acquire the target object features. These target object features can be the object features extracted from the target template (i.e., the template of the target object). Based on the feature similarity between the features to be matched and the target object features, the degree of matching between the object to be identified and the target object can be determined. There is a positive correlation between feature similarity and object matching degree; the higher the feature similarity, the higher the object matching degree; conversely, the lower the feature similarity, the lower the object matching degree.

[0119] In one exemplary embodiment, the above-described apparatus further includes:

[0120] The fourth determining unit is used to determine the target overlap between the detection box to be identified and the target detection box;

[0121] The fifth determining unit is used to determine that the detection box to be identified does not match the target detection box when the target overlap is less than or equal to the first set threshold.

[0122] In one exemplary embodiment, the above-described apparatus further includes:

[0123] The sixth determining unit is used to determine that the detection box to be identified does not match the target detection box when the number of times the target detection box has been used for object matching is greater than a second set threshold.

[0124] In one exemplary embodiment, the third determining unit includes:

[0125] The first determining module is used to determine the feature similarity between the feature to be matched and the target object feature of the target object;

[0126] The second determining module is used to determine whether the object to be identified matches the target object when the feature similarity is greater than or equal to the third set threshold.

[0127] The third determination module is used to determine whether the object to be identified matches the target object when the feature similarity is less than a third set threshold.

[0128] In one exemplary embodiment, the above-described apparatus further includes:

[0129] The update unit is used to update the target detection box using the target detection box after determining the detection box to be recognized and the target detection box corresponding to the image to be recognized, and if it is determined that the object to be recognized matches the target object, so as to obtain the updated target detection box.

[0130] In one exemplary embodiment, the above-described apparatus further includes:

[0131] The seventh determining unit is used to update the target detection box using the detection box to be identified, and after obtaining the updated target detection box, use the depth image matched with the image to be identified to determine the position information of the target object in the target detection box.

[0132] The control unit is used to control the following device of the target object to move according to the position information of the target object.

[0133] In one exemplary embodiment, the above-described apparatus further includes:

[0134] The matching unit is used to perform position matching between the image to be recognized and the depth image before determining the position information of the target object in the target detection box using the depth image matched with the image to be recognized, so as to obtain position mapping information between the image to be recognized and the depth image. The position mapping information is used to perform position mapping between the image to be recognized and the depth image.

[0135] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0136] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute program code for any of the matching methods described above in the embodiments of this application.

[0137] Optionally, in this embodiment, the storage medium may be located on at least one of the multiple network devices in the network shown in the above embodiment.

[0138] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0139] S1, determine the detection box to be identified and the target detection box corresponding to the image to be identified, wherein the detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object;

[0140] S2, if it is determined that the detection box to be identified does not match the target detection box, determine the matching features of the object to be identified in the image to be identified;

[0141] S3, determine the matching degree between the object to be identified and the target object based on the features to be matched.

[0142] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0143] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0144] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described matching method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0145] Figure 5 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506, and a communication bus 508. The processor 502, communication interface 504, and memory 506 communicate with each other via the communication bus 508.

[0146] Memory 506 is used to store computer programs;

[0147] When processor 502 executes a computer program stored in memory 506, it performs the following steps:

[0148] S1, determine the detection box to be identified and the target detection box corresponding to the image to be identified, wherein the detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object;

[0149] S2, if it is determined that the detection box to be identified does not match the target detection box, determine the matching features of the object to be identified in the image to be identified;

[0150] S3, determine the matching degree between the object to be identified and the target object based on the features to be matched.

[0151] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 5 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0152] The aforementioned memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0153] As an example, the memory 506 described above may include, but is not limited to, the first determining unit 402, the second determining unit 404, and the third determining unit 406 of the object matching device. Furthermore, it may include, but is not limited to, other module units of the object matching device, which will not be elaborated upon in this example.

[0154] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0155] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0156] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. The device implementing the above matching method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, PDA, mobile Internet Devices (MID), PAD, etc. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0157] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0158] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0160] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A matching method, characterized in that, The method includes: Determine the detection box to be identified and the target detection box corresponding to the image to be identified, wherein the detection box to be identified contains the object to be identified, and the target detection box is determined based on the target object; If it is determined that the detection box to be identified does not match the target detection box, the matching features of the object to be identified in the image to be identified are determined; and the matching degree between the object to be identified and the target object is determined based on the matching features. If it is determined that the object to be identified matches the target object, the target detection box is updated using the detection box to be identified to obtain the updated target detection box, and the target object is updated using the object to be identified to obtain the updated target object, wherein the number of the target objects stored is one; the position information of the target object in the target detection box is determined using a depth image matched with the image to be identified; the following device of the target object is controlled to follow the target object according to the position information of the target object, wherein the more objects to be identified contained in the image to be identified, the closer the distance between the following device and the target object; The method further includes: when the number of objects to be identified in the image to be identified is greater than one and the number of consecutive box matching exceeds the maximum number of matching times, calling features for matching, and deleting the target detection boxes that have been matched more than the valid number of times consecutively.

2. The method according to claim 1, characterized in that, The step of determining that the detection box to be identified does not match the target detection box includes: Determine the target overlap between the detection box to be identified and the target detection box; If the target overlap is less than or equal to a first set threshold, it is determined that the detection box to be identified does not match the target detection box.

3. The method according to claim 1, characterized in that, The step of determining that the detection box to be identified does not match the target detection box includes: If the number of times the target detection box has been used for object matching exceeds a second preset threshold, it is determined that the detection box to be identified does not match the target detection box.

4. The method according to claim 1, characterized in that, Determining the matching degree between the object to be identified and the target object based on the features to be matched includes: Determine the feature similarity between the feature to be matched and the target object feature of the target object; If the feature similarity is greater than or equal to a third preset threshold, it is determined that the object to be identified matches the target object; If the feature similarity is less than the third preset threshold, it is determined that the object to be identified does not match the target object.

5. The method according to claim 1, characterized in that, Before determining the location information of the target object within the target detection box using a depth image matched with the image to be identified, the method further includes: The image to be identified is matched with the depth image to obtain position mapping information between the image to be identified and the depth image, wherein the position mapping information is used to perform position mapping between the image to be identified and the depth image.

6. A matching device, characterized in that, The device includes: The first determining unit is used to determine the detection box to be identified and the target detection box corresponding to the image to be identified, wherein the detection box to be identified contains the detected object to be identified, and the target detection box is determined based on the target object; The second determining unit is used to determine the matching features of the object to be identified in the image to be identified when it is determined that the detection box to be identified does not match the target detection box; The third determining unit is used to determine the matching degree between the object to be identified and the target object based on the features to be matched; The device further includes: an update unit, configured to update the target detection box using the target detection box after determining the target detection box and the target detection box corresponding to the image to be identified, and if it is determined that the target object matches the target object, to obtain the updated target detection box; a seventh determination unit, configured to determine the position information of the target object in the target detection box using a depth image that matches the image to be identified; and a control unit, configured to control the target object following device to move according to the position information of the target object, wherein the more targets to be identified contained in the image to be identified, the closer the distance between the following device and the target object; The device is further configured to: when it is determined that the object to be identified matches the target object, update the target object using the object to be identified to obtain the updated target object, wherein the number of the stored target objects is one; The device is further configured to: when the number of objects to be identified in the image to be identified is greater than one and the number of consecutive box matching exceeds the maximum number of matching times, invoke features for matching and delete the target detection boxes that have been matched more than the valid number of times.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 5 through the computer program.

Images