Object detection method and related device

Abstract

Embodiments of the present application disclose an object detection method and related equipment, wherein the method comprises: obtaining position information of P configuration key points in pre-configuration information of an object to be detected and position information of Q detection key points in an image of the object to be detected; obtaining M key point pairs based on the position information of the P configuration key points and the position information of the Q detection key points; determining a matching score of each key point pair based on the pre-configuration information; determining N key point pairs from the M key point pairs based on the matching score of each key point pair, and the matching score of any key point pair in the N key point pairs is greater than or equal to a preset score threshold; and obtaining position information of a target object in the object to be detected based on the position information of the N key point pairs. The embodiments of the present application are beneficial to improving the effectiveness and accuracy of object detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to an object detection method and related equipment. Background Technology

[0002] With the development of artificial intelligence and computer vision technologies, target detection and tracking technologies are widely used in industrial inspection and maintenance scenarios. By analyzing and processing acquired images, target objects in the images can be automatically located, which is of great significance in simplifying manual inspection work, reducing labor costs, and improving maintenance efficiency. Currently, deep neural network-based methods are the mainstream target detection methods. Their operation can be summarized as follows: the acquired and preprocessed image is input into a trained deep neural network model; the deep neural network model obtains the output result through inference; and the output result is post-processed to obtain the category and location of the target object in the image.

[0003] Deep neural network-based object detection technology requires relatively clear and complete object images as input to obtain sufficient feature maps for object classification and localization. However, in real-world business scenarios, objects or target components on objects are often completely occluded or have too low resolution, which leads to a decrease in the effectiveness and accuracy of detection. Summary of the Invention

[0004] This application provides an object detection method and related equipment, which helps to improve the effectiveness and accuracy of object detection.

[0005] In a first aspect, embodiments of this application provide an object detection method, the method comprising:

[0006] The pre-configuration information of the object to be detected and the detection result of the image of the object to be detected are obtained. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection result includes the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold.

[0007] Based on the location information of P configuration key points and Q detection key points, M key point pairs are obtained. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to a preset number threshold.

[0008] The matching score of each key point pair is determined based on pre-configured information;

[0009] Based on the matching scores of each keypoint pair, N keypoint pairs are determined from M keypoint pairs. The matching score of any keypoint pair among the N keypoint pairs is greater than or equal to a preset score threshold, and N is greater than or equal to a preset quantity threshold.

[0010] Based on the position information of N key point pairs, the position information of the target object in the object to be detected is obtained.

[0011] In this embodiment, the pre-configured information can be the layout information of some objects in the object to be detected. Specifically, it can be the position information of these objects. For example, in telecommunications equipment, the position information of the four corners of the panel, the position information of the center points of single and double screws, and the position information of the center point of the cable socket; or in a vehicle, the position information of the four corners of the license plate and the position information of the center point of the license plate, etc. Here, the configuration key points refer to the pre-configured key positions in the object to be detected used to locate and map the target object (e.g., the cable socket in the telecommunications equipment). For example, the center points of P objects (e.g., the single and double screws in the telecommunications equipment). The detection key points refer to the center points of Q objects detected in the image of the object to be detected (e.g., the single and double screws detected in the image of the telecommunications equipment). The categories of the P objects and the Q objects are usually corresponding. Based on the location information of P configuration keypoints and Q detection keypoints, the Q detection keypoints are matched with the P configuration keypoints to obtain the detection keypoints with the highest matching degree. The resulting M keypoint pairs are all accurate and stable keypoint pairs. For the M accurate and stable keypoint pairs, the final matching score of each keypoint pair is calculated based on the pre-configured information. The keypoint pairs with matching scores greater than or equal to a preset threshold are selected. This yields the N keypoint pairs with the best matching and most favorable conditions for localization mapping. Based on the N keypoint pairs with the best matching and most favorable conditions for localization mapping, the target object in the object to be detected is mapped to the corresponding image. This is beneficial for accurately and effectively detecting the position of the target object in the image and is applicable to target objects with occlusion and low resolution, thereby improving the effectiveness and accuracy of object detection.

[0012] In one possible implementation, the detection result also includes the detection score of each of the Q detection keypoints. Based on the location information of P configuration keypoints and the location information of the Q detection keypoints, M keypoint pairs are obtained, including:

[0013] Based on the location information of P configuration key points and Q detection key points, the positioning distance between each detection key point and each configuration key point among the P configuration key points is obtained.

[0014] Based on the positioning distance between each detection key point and each configuration key point, and the detection score of each detection key point, M key point pairs are obtained.

[0015] In this embodiment, the positioning distance refers to the distance between the relative coordinates of the configured key point relative to the reference point in the preset configuration information and the relative coordinates of the detected key point relative to the reference point in the image. For each detected key point and each pre-configured key point, the positioning distance between each detected key point and each configured key point is obtained based on its position information. At the same time, combined with the detection score of the detected key point, each detected key point is matched with each configured key point. This can solve the mismatch problem caused by relying solely on the positioning distance for matching, thereby facilitating the matching of M key point pairs accurately and stably.

[0016] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to the same position in the object to be detected, based on the position information of P configuration keypoints and Q detection keypoints, the localization distance between each detection keypoint and each of the P configuration keypoints is obtained, including:

[0017] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0018] Based on the location information of the detection reference point and the location information of Q detection key points, obtain the Q second relative coordinates of the Q detection key points relative to the detection reference point;

[0019] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q second relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0020] In this embodiment, the configuration reference point refers to a reference point in the preset configuration information, such as any one of the four corners of the panel in a telecommunications device. The detection reference point refers to a reference point detected in the image of the object to be detected, such as any one of the four corners of the detection frame of the panel in the image of the telecommunications device. Since the first relative coordinate represents the positional relationship (or layout relationship) of the configuration key point relative to the configuration reference point, and the second relative coordinate represents the positional relationship (or layout relationship) of the detection key point relative to the detection reference point, when the configuration reference point and the detection reference point correspond to the same position in the object to be detected, such as the lower left corner of the panel of the telecommunications device, the Euclidean distance between the first relative coordinate and the second relative coordinate can represent the similarity of the configuration key point and the detection key point in the layout position. That is to say, the positioning distance between the detection key point and the configuration key point includes the similarity of the detection key point and the configuration key point in the layout position. Using the positioning distance between each detection key point and each configuration key point as a reference index for key point pair matching is beneficial to matching key point pairs that are more similar in layout position, which to a certain extent helps to improve the accuracy and stability of the matched M key point pairs.

[0021] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to different positions in the object to be detected, based on the position information of P configuration keypoints and Q detection keypoints, the localization distance between each detection keypoint and each of the P configuration keypoints is obtained, including:

[0022] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0023] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0024] Based on the location information of Q key detection points, the location information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configured reference point, obtain the Q third relative coordinates of the Q key detection points with respect to the configured reference point;

[0025] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q third relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0026] In this embodiment, when the configuration reference point and the detection reference point correspond to different positions in the object to be detected, such as the configuration reference point corresponding to the lower left corner of the panel of the telecommunications equipment and the detection reference point corresponding to the upper right corner of the detection frame of the panel of the telecommunications equipment, it is necessary to obtain the relative coordinates of the detection reference point relative to the configuration reference point (or the relative coordinates of the configuration reference point relative to the detection reference point). Based on the position information of the detection key point, the position information of the detection reference point, and the relative coordinates of the detection reference point relative to the configuration reference point, the third relative coordinate of each of the Q detection key points relative to the configuration reference point is calculated. Then, the Euclidean distance between the first relative coordinate and the third relative coordinate is calculated. When obtaining the positioning distance between each detection key point and each configuration key point, the calculation of the relative coordinates is transformed to the reference point at the same position in the device to be detected. This unifies the calculation standard of the relative coordinates, which helps to avoid misjudging the positioning distance between detection key points and configuration key points that are actually close as being far apart, and vice versa. This can reduce the mismatch rate between detection key points and configuration key points.

[0027] In one possible implementation, based on the positioning distance between each detected keypoint and each configured keypoint, and the detection score of each detected keypoint, M keypoint pairs are obtained, including:

[0028] The maximum positioning distance A is determined from the Q positioning distances corresponding to each configuration key point;

[0029] The matching degree between each detection key point and each configuration key point is calculated using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A.

[0030] For each configuration key point, the detection key point with the highest matching degree is determined from Q detection key points, and the configuration key point and the detection key point with the highest matching degree are determined as key point pairs to obtain M key point pairs.

[0031] In this embodiment, when matching the detected key points with the configured key points, not only the positioning distance between the detected key points and the configured key points is considered, but also the detection score of the detected key points is included. The matching degree between each detected key point and each configured key point is calculated using the detection score, the positioning distance, and the maximum positioning distance (i.e., the maximum positioning distance A) among the Q positioning distances corresponding to each configured key point. The detected key point with the highest matching degree and the configured key point are taken as the key point pair. Since the detection score of the detected key points is considered during matching, it is beneficial to reduce the situation of mismatch, thereby helping to obtain M key point pairs with high accuracy and stability.

[0032] In one possible implementation, the pre-configured information further includes category information for each of the P configuration key points, and the detection results further include the detection score and category information for each of the Q detection key points. Based on the location information of the P configuration key points and the location information of the Q detection key points, M key point pairs are obtained, including:

[0033] Based on the category information of each configuration key point and the category information of each detection key point, the configuration key points and detection key points of the same category among the P configuration key points and Q detection key points are determined as key point sets, so as to obtain at least one key point set of the same type;

[0034] For any set of key points H in at least one set of key points of the same type, based on the location information of R configuration key points and S detection key points in any set of key points H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained; R is greater than or equal to 1 and less than P; S is greater than or equal to 1 and less than Q.

[0035] Based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, K key point pairs are obtained. M key point pairs include K key point pairs, where K is greater than or equal to 0 and less than or equal to R.

[0036] In this embodiment, when matching Q detection keypoints with P configuration keypoints, the P configuration keypoints and Q detection keypoints can be categorized based on the category information of each configuration keypoint and each detection keypoint to obtain at least one set of keypoints of the same type. For example, the keypoints corresponding to a single screw in the pre-configuration information of a telecommunications device and a single screw in an image belong to the same set of keypoints, as do the keypoints corresponding to a double screw in the pre-configuration information of a telecommunications device and a double screw in an image, and so on. Based on obtaining at least one set of keypoints of the same type, for any set of keypoints of the same type H, it is only necessary to match S detection keypoints with R configuration keypoints in any set of keypoints of the same type H based on the positioning distance between each detection keypoint A and each configuration keypoint B and the detection score of each detection keypoint A to obtain K keypoint pairs, without the need for inter-category matching. This not only solves the mismatch problem caused by matching based solely on positioning distance, matching M accurate and stable keypoint pairs, but also relatively reduces the amount of computation and matching.

[0037] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result also includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to the same position in the object to be detected, based on the position information of R configuration key points and S detection key points in any set of key points of the same type H, the localization distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained, including:

[0038] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0039] Based on the location information of the detection reference point and the location information of S detection key points, obtain the S fifth relative coordinates of the S detection key points relative to the detection reference point;

[0040] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S fifth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0041] In this embodiment of the application, it should be understood that for any set of key points H of the same type, since only intra-category matching is required, when the configuration reference point and the detection reference point correspond to the same position in the object to be detected, such as the lower left corner of the panel of a telecommunications device, calculating the positioning distance only requires calculating the R fourth relative coordinates of the R configuration key points relative to the configuration reference point, and only requires calculating the S fifth relative coordinates of the S detection key points relative to the detection reference point. Then, the Euclidean distance between each fourth relative coordinate and each fifth relative coordinate is calculated, which yields the positioning distance between each detection key point A and each configuration key point B. Using the positioning distance between each detection key point A and each configuration key point B as a reference index for intra-category key point pair matching is beneficial for matching key point pairs that are more similar in layout position within the same category. Moreover, since the positioning distance is calculated only within the category, the amount of computation is greatly reduced.

[0042] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result also includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to different positions in the object to be detected, based on the position information of R configuration key points and S detection key points in any set of key points of the same type H, the localization distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained, including:

[0043] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0044] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0045] Based on the location information of S detection key points, the location information of detection reference points, and the relative coordinates of detection reference points with respect to configuration reference points, obtain the S sixth relative coordinates of the S detection key points with respect to the configuration reference points;

[0046] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S sixth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0047] In this embodiment, when the configuration reference point and the detection reference point correspond to different positions in the object to be detected, for example, the configuration reference point corresponds to the lower left corner of the panel of the telecommunications equipment, and the detection reference point corresponds to the upper right corner of the detection frame of the panel of the telecommunications equipment, for any set of key points of the same type H, when performing intra-class matching, it is necessary to obtain the relative coordinates of the detection reference point with respect to the configuration reference point (or the relative coordinates of the configuration reference point with respect to the detection reference point). Based on the position information of the detection key point, the position information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configuration reference point, the relative coordinates of each detection key point among the S detection key points with respect to the configuration reference point are calculated. The sixth relative coordinate of the reference point is used, and then the Euclidean distance between the fourth and sixth relative coordinates is calculated. When obtaining the positioning distance between each detection key point and each configuration key point in any set of similar key points H, the calculation of relative coordinates is transformed to the reference point at the same position in the device to be tested. This standardization of relative coordinate calculation helps to avoid misjudging detection key points and configuration key points that are actually close in positioning distance as far in positioning distance, and vice versa. This can reduce the mismatch rate between detection key points and configuration key points.

[0048] In one possible implementation, based on the positioning distance between each detected keypoint A and each configured keypoint B, and the detection score of each detected keypoint A, K keypoint pairs are obtained, including:

[0049] The maximum positioning distance B is determined from the S positioning distances corresponding to each configuration key point B;

[0050] The matching degree between each detection key point A and each configuration key point B is calculated using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and the maximum positioning distance B.

[0051] For each configuration key point B, the detection key point A with the highest matching degree with configuration key point B is determined from S detection key points. The configuration key point B and the detection key point A with the highest matching degree are determined as key point pairs to obtain K key point pairs.

[0052] In this embodiment, when matching the detected key point A with the configured key point B, not only is the positioning distance between the detected key point A and the configured key point B considered, but the detection score of the detected key point A is also included. The matching degree between each detected key point A and each configured key point B is calculated using the detection score, the positioning distance, and the maximum positioning distance (i.e., the maximum positioning distance B) among the S positioning distances corresponding to each configured key point B. The detected key point A with the highest matching degree and the configured key point B are taken as the key point pair. Since the detection score of the detected key point A is considered during matching, it is beneficial to reduce the situation of mismatch within the same category, thereby helping to obtain K key point pairs with high accuracy and stability.

[0053] In one possible implementation, the pre-configuration information also includes the position information of the target point of the target object in the object to be detected, and the matching score of each key point pair is determined based on the pre-configuration information, including:

[0054] Based on the location information of the target point and the location information of the configuration key points in each key point pair, the target distance between the configuration key point in each key point pair and the target point is calculated to obtain M target distances;

[0055] Determine the maximum target distance from M target distances;

[0056] The matching score of each keypoint pair is calculated by using the matching degree between the configured keypoint and the detected keypoint, the target distance between the configured keypoint and the target point, and the maximum target distance.

[0057] In this embodiment, the target point refers to the center point of the target object in the object to be detected. For the M matched key point pairs, the target distance between the configured key point and the target point in each key point pair is calculated. Then, the matching score of each key point pair is calculated using the matching degree between the configured key point and the detected key point, the target distance between the configured key point and the target point, and the maximum target distance. In this way, when calculating the matching score of each key point pair, in addition to the original matching degree of each key point pair (considering the detection score and the positioning distance), the target distance that can represent the positional relationship (or layout relationship) between the configured key point and the target point in the object to be detected is also considered. The N key point pairs selected based on the detection score, positioning distance, and target distance are the best matched key point pairs. Using the best matched N key point pairs to map the target point helps to reduce the mapping deviation of the target point, thereby making the positional information of the target object in the image more accurate.

[0058] In one possible implementation, the pre-configured information also includes the position information of the target point of the target object in the object to be detected. Based on the position information of N key point pairs, the position information of the target object in the object to be detected is obtained, including:

[0059] Based on the location information of N key point pairs, a mapping matrix is ​​calculated from the N configured key points in the N key point pairs to the N detected key points in the N key point pairs;

[0060] The target points are mapped onto the image using a mapping matrix to obtain the location information of the target object.

[0061] In this embodiment of the application, it should be understood that since the N key point pairs are the best-matched key point pairs, the mapping matrix calculated based on the position information of the best-matched key point pairs, from the N configuration key points in the N key point pairs to the N detection key points in the N key point pairs, performs better in terms of stability and robustness. Using this mapping matrix to map the target points into the image is beneficial to improving the accuracy of obtaining the position of the target object in the image when detecting objects. In addition, it can also effectively detect the position of the target object in the image for target objects that are occluded or have low resolution.

[0062] Secondly, embodiments of this application provide an object detection device, which includes an acquisition unit and a processing unit, wherein:

[0063] The acquisition unit is used to acquire pre-configuration information of the object to be detected and to acquire the detection result of the image of the object to be detected. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection result includes the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold.

[0064] The processing unit is used to obtain M key point pairs based on the location information of P configuration key points and the location information of Q detection key points. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to a preset number threshold.

[0065] The processing unit is also used to determine the matching score of each key point pair based on pre-configured information;

[0066] The processing unit is also used to determine N key point pairs from M key point pairs based on the matching scores of each key point pair, wherein the matching score of any key point pair among the N key point pairs is greater than or equal to a preset score threshold, and N is greater than or equal to a preset quantity threshold.

[0067] The processing unit is also used to obtain the position information of the target object in the object to be detected based on the position information of N key point pairs.

[0068] It should be noted that the second aspect is the apparatus corresponding to the first aspect above, used to implement the various methods and steps provided in the first aspect. For specific implementation details and beneficial effects, please refer to the first aspect above.

[0069] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and one or more programs. The processor is connected to the memory, and the one or more programs are stored in the memory and configured to implement the method in the first aspect described above when executed by the processor.

[0070] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for execution by a device, wherein the computer program, when executed, implements the method described in the first aspect above.

[0071] Fifthly, embodiments of this application provide a computer program product that, when run by an electronic device, causes the electronic device to perform the method described in the first aspect. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0073] Figure 1 A schematic diagram of the core module of a deep neural network-based object detection technology proposed in a related field;

[0074] Figure 2 This is a schematic diagram illustrating an application scenario of an object detection method provided in an embodiment of this application;

[0075] Figure 3 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0076] Figure 4A This application provides a schematic diagram of the layout of objects in a telecommunications device.

[0077] Figure 4B This is a schematic diagram illustrating the pre-configuration information of various objects in a telecommunications device, as provided in an embodiment of this application.

[0078] Figure 4C A schematic diagram illustrating the detection results of various objects in a telecommunications device provided in an embodiment of this application;

[0079] Figure 5 This application provides a schematic diagram illustrating the matching of detection key points and configuration key points in an embodiment.

[0080] Figure 6 A schematic diagram illustrating an optimal keypoint pair provided in an embodiment of this application;

[0081] Figure 7 This application provides a schematic diagram of mapping target points onto an image of an object to be detected, as provided in an embodiment of the present application.

[0082] Figure 8 This application provides an example of a non-frontal image capture of an object to be detected;

[0083] Figure 9 A flowchart illustrating another object detection method provided in an embodiment of this application;

[0084] Figure 10 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application;

[0085] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0086] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0087] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0088] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, an application running on a terminal device and the terminal device can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0089] First, the relevant terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0090] (1) Pre-configuration information: layout information of some objects in the object to be detected, specifically the coordinate information of key points, reference points and target points in the object to be detected. Among them, key points, reference points and target points usually refer to the center points of objects in the object to be detected.

[0091] (2) Key Points: Key location points in the pre-configured information and the image of the object to be detected, used to locate and map the target object.

[0092] (3) Key Point Pair: A pair of key points that match in the image of the object to be detected, based on the pre-configured information.

[0093] (4) Reference Points: Reference position points in the pre-configured information and the image of the object to be detected, used to calculate the relative coordinates.

[0094] (5) Target Points: Points to be located and tracked in the pre-configured information, usually referring to the center point of the target object in the object to be detected.

[0095] (6) Location Distance: The distance between the relative coordinates of the key points in the pre-configured information relative to the reference point and the relative coordinates of the key points in the image of the object to be detected relative to the reference point.

[0096] (7) Detection Score: The score of the key point detection results in the image of the object to be detected.

[0097] (8) Target Distance: The distance between the key point in the pre-configured information and the target point, or the distance between the relative coordinates of the key point with respect to the reference point and the relative coordinates of the target point with respect to the reference point.

[0098] To facilitate understanding of the embodiments of this application, further analyze and propose the specific technical problems to be solved by this application, the relevant technical solutions of this application are briefly introduced below.

[0099] Please see Figure 1 , Figure 1 This is a schematic diagram of the core module of a deep neural network-based object detection technology proposed in a related field, as shown below. Figure 1 As shown, it mainly includes: a target image acquisition module, an object detection result acquisition module, a training sample acquisition module, a model training module, a loss acquisition module, and a target object detection model acquisition module. The target image acquisition module is used to acquire the image to be detected. The object detection result acquisition module is used to input the target image into the target object detection model for object detection, obtaining the object detection result of the target image. The target object detection model is obtained using the training sample acquisition module, the model training module, the loss acquisition module, and the target object detection model acquisition module. The training sample acquisition module is used to acquire training samples. The model training module is used to input the training samples into the object detection model for model training. The object detection model includes a detection module, a classification module, and a discriminant module. The loss acquisition module is used to obtain the detection loss generated by the detection module, the classification loss generated by the classification module, and the discriminant loss generated by the discriminant module during model training. The target object detection model acquisition module is used to update the object detection model based on the detection loss, classification loss, and discriminant loss to obtain the target object detection model.

[0100] It can be seen that, Figure 1In the deep neural network-based object detection scheme shown, the object detection result acquisition module requires a clear, complete image of the target object without occlusion as input to obtain sufficient features of the target object through the deep neural network model, and then classify and locate the target object. However, when the target object is occluded or has too low resolution, such as when the cable socket in telecommunications equipment is usually obscured by cables, in such scenarios... Figure 1 The object detection scheme based on deep neural networks shown is difficult to effectively detect the position of the target object in the object.

[0101] Based on the defects and shortcomings of related technologies, this application proposes an object detection method, which can be based on... Figure 2 The application environment implementation shown is as follows: Figure 2 As shown, the administrator configures the category information (label) and layout information of multiple objects in the object to be detected in the management system. The layout information usually refers to the position information. The multiple objects include key points, reference points and target points in the object to be detected.

[0102] The location information of key points, reference points and target points is represented by a Cartesian coordinate system x0y, where x is the horizontal coordinate and y is the vertical coordinate. The origin 0 can be any point in the object to be detected, such as the upper left corner of the panel in a telecommunications device.

[0103] Due to the requirements of calculating the mapping matrix, at least four key points need to be configured. The selection criteria should meet the following conditions: they should be clear and complete within the object to be detected, not occluded, and easily detected by the machine from the image. For example, the center points of single and double screws in telecommunications equipment. Of course, the selection of key points will vary depending on the object to be detected. Key points can be represented in the pre-configured information in the following form:

[0104] Key points:

[0105] label1:[(x1,y1),…]

[0106] label2:[(x2,y2),…]

[0107] Here, label1 and label2 represent the categories of key points, (x1, y1)... represent the location information of key points in category 1 in the object to be detected, and (x2, y2),... represent the location information of key points in category 2 in the object to be detected.

[0108] Since the reference point is only used for calculating relative coordinates, only one reference point is needed. For example, it can be any point among the four corners of the panel of a telecommunications device. The selection criteria should also meet the following requirements: it should be clear and complete in the object to be detected, not obstructed, and easily detected by the machine from the image. The reference point can be represented in the pre-configured information in the following form:

[0109] Reference point:

[0110] label3:(x3,y3), where label3 represents the category of the reference point, and (x3,y3) represents the position information of the reference point in the object to be detected.

[0111] The target point can be represented in the pre-configured information in the following form:

[0112] Target point:

[0113] label4:[(x4,y4),…]

[0114] Here, label4 represents the category of the target point, (x4, y4), ... represent the position information of the target point in the object to be detected, and the number of target points can be one or more.

[0115] In addition, administrators also need to configure the path of the object detection model in the management system and the category information of the objects to be detected when inferring from the image of the object to be detected. For example, the category information could be a single screw, double screws, etc. in the image of telecommunications equipment. Figure 2 The management system configuration page shown allows administrators to configure the category and location information of key points, reference points, and target points, as well as the path of the object detection model and the category information of the objects to be detected. After configuring the relevant information, the administrator pushes the object detection model to the user device and deploys it on a service device, which can be any server, distributed system, or cloud. Upon receiving the object detection model, the user device can deploy it locally or update an older version of the detection model based on user actions. In application scenarios, users can collect images or videos of the objects to be detected. For these images or videos, such as... Figure 2 The user device operation interface shown in the figure allows users to choose to perform object detection directly on their device (i.e., local detection) if the user device or the detection system running on the user device stores pre-configured information of the object to be detected. Alternatively, users can choose to upload images or videos of the object to be detected to the server device for object detection (i.e., upload detection).

[0116] The object detection method and related equipment provided in the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0117] Please see Figure 3 , Figure 3 This is a flowchart illustrating an object detection method provided in an embodiment of this application. This method can be applied to... Figure 2 In the scenario shown, the specific actions can be performed by electronic devices, such as... Figure 3 As shown, the method may include steps 301-305:

[0118] 301: Obtain the pre-configuration information of the object to be detected and obtain the detection results of the image of the object to be detected. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection results include the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold.

[0119] In this embodiment, the pre-configured information may be the layout information of some objects in the object to be detected, specifically the position information of these objects, such as... Figure 4A The telecommunications equipment shown includes the position information of the four corners of the panel, the position information of the center points of single and double screws, and the position information of the center point of the cable connector. Corresponding to... Figure 4A The telecommunications equipment shown uses the center points of single and double screws as key points, any one of the four corners of the panel as reference points, and the center point of the cable connector as the target point. The position information of the key points, reference points, and target points are pre-configured to obtain pre-configured information. The key points in the pre-configured information are called configuration key points. For example, there can be P configuration key points, the number of which varies depending on the object to be detected. The reference points in the pre-configured information are called configuration reference points. For example, only one configuration reference point is needed, such as the lower left corner of the panel frame. The target point varies depending on the target object within the object to be detected. Figure 4A The 30 cable ports correspond to... Figure 4A The distribution of pre-configured information for the telecommunications equipment shown can be as follows: Figure 4B As shown, Figure 4B Any point within a solid square can be used as a configuration reference point, solid dots represent configuration key points, and hollow squares represent target objects.

[0120] When an electronic device receives an image of an object to be detected, it acquires and parses pre-configured information about the object, and then uses an object detection model to detect the image of the object to obtain a detection result. For example, since the detection categories of the object detection model are pre-configured, the detection result may include the location information of Q detection keypoints in the image, the detection score of each keypoint, and the location information and detection score of a detection reference point. For telecommunications equipment, the detection result may also include the panel's location information and detection score. For example, Figure 4C As shown, the location information of the Q detection keypoints and the panel is presented in the image as detection boxes. Each detection box includes its minimum horizontal coordinate (left), maximum horizontal coordinate (right), bottom vertical coordinate (bottom), top vertical coordinate (top), the category information of the object contained in the detection box, and the detection score of that object. For a detection box whose category information is a keypoint, the coordinates of its center point are calculated using the minimum horizontal coordinate (left), maximum horizontal coordinate (right), bottom vertical coordinate (bottom), and top vertical coordinate (top), thus obtaining the location information of the detection keypoint. For the detection box of the panel, the coordinates of any one of its four corners are used as the location information of the detection reference point.

[0121] For example, since the calculation of the subsequent mapping matrix requires at least 4 configuration key points and at least 4 detection key points, the preset quantity threshold can be 4, that is, both P and Q must be greater than or equal to 4.

[0122] 302: Based on the location information of P configuration key points and Q detection key points, M key point pairs are obtained. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to a preset number threshold.

[0123] In this embodiment, the electronic device obtains the positioning distance between each detection key point and each of the P configuration key points based on the location information of P configuration key points and Q detection key points. Then, based on the positioning distance between each detection key point and each configuration key point, and the detection score of each detection key point, M key point pairs are obtained. In this embodiment, for each detected key point and each pre-configured configuration key point, the positioning distance between each detection key point and each configuration key point is obtained based on its location information. Simultaneously, by combining the detection score of the detection key point, each detection key point is matched with each configuration key point. This solves the mismatch problem caused by relying solely on positioning distance for matching, thus facilitating the matching of accurate and stable M key point pairs. Similarly, based on the need to calculate the mapping matrix, M must be greater than or equal to 4.

[0124] For example, when the configuration reference point and the detection reference point correspond to the same location in the object to be detected, such as the lower left corner of the panel of a telecommunications device (the detection reference point is the lower left corner of the detection frame of the panel), based on the position information of P configuration key points and Q detection key points, the positioning distance between each detection key point and each of the P configuration key points is obtained, including:

[0125] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0126] Based on the location information of the detection reference point and the location information of Q detection key points, obtain the Q second relative coordinates of the Q detection key points relative to the detection reference point;

[0127] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q second relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0128] In this embodiment, the relative coordinates of each configuration keypoint relative to the configuration reference point are called the first relative coordinates. Therefore, P configuration keypoints will have P first relative coordinates. The relative coordinates of each detection keypoint relative to the detection reference point are called the second relative coordinates. Therefore, Q detection keypoints will have Q second relative coordinates. The positioning distance between each configuration keypoint and each detection keypoint is obtained by calculating the Euclidean distance between each first relative coordinate and each second relative coordinate. For example, if the first relative coordinate of configuration keypoint p relative to the configuration reference point is (xp, yp), and the second relative coordinate of detection keypoint q relative to the detection reference point is (xq, yq), then the Euclidean distance between (xp, yp) and (xq, yq) represents the positioning distance between configuration keypoint p and detection keypoint q. In this embodiment, since the first relative coordinate represents the positional relationship between the configured key point and the configured reference point, and the second relative coordinate represents the positional relationship between the detected key point and the detected reference point, when the configured reference point and the detected reference point correspond to the same position in the object to be detected, the Euclidean distance between the first relative coordinate and the second relative coordinate is calculated. This distance can represent the similarity between the configured key point and the detected key point in the layout position. In other words, the positioning distance between the detected key point and the configured key point includes the similarity between the detected key point and the configured key point in the layout position. Using the positioning distance between each detected key point and each configured key point as a reference index for key point pair matching is beneficial to matching key point pairs that are more similar in layout position, which to some extent helps to improve the accuracy and stability of the matched M key point pairs.

[0129] For example, when the configuration reference point and the detection reference point correspond to different positions in the object to be detected, such as the configuration reference point corresponding to the lower left corner of the panel of a telecommunications device and the detection reference point corresponding to the upper right corner of the detection frame of the panel of the telecommunications device, based on the position information of P configuration key points and Q detection key points, the positioning distance between each detection key point and each of the P configuration key points is obtained, including:

[0130] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0131] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0132] Based on the location information of Q key detection points, the location information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configured reference point, obtain the Q third relative coordinates of the Q key detection points with respect to the configured reference point;

[0133] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q third relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0134] In this embodiment, when the configuration reference point and the detection reference point correspond to different positions in the object to be detected, directly using the Euclidean distance between the first relative coordinate and the second relative coordinate as the positioning distance between the detection key point and the configuration key point may lead to detection key points and configuration key points that are actually closer being incorrectly identified as having a greater positioning distance, and vice versa, resulting in mismatch. Therefore, it is necessary to obtain the relative coordinates of the detection reference point relative to the configuration reference point (or the relative coordinates of the configuration reference point relative to the detection reference point). Based on the position information of the detection key point, the position information of the detection reference point, and the relative coordinates of the detection reference point relative to the configuration reference point, the relative coordinates of each detection key point relative to the configuration reference point, i.e., the third relative coordinate, are calculated. Then, the Euclidean distance between the first relative coordinate and the third relative coordinate is calculated to obtain the positioning distance between each detection key point and each configuration key point. In short, when obtaining the positioning distance between each detection key point and each configuration key point, the calculation of relative coordinates needs to be transformed to a reference point relative to the same position in the device under test. This standardization of relative coordinate calculation helps to avoid misjudging detection key points and configuration key points that are actually close in positioning distance as far in positioning distance, and vice versa. This reduces the mismatch rate between detection key points and configuration key points.

[0135] For example, based on the positioning distance between each detection keypoint and each configuration keypoint, and the detection score of each detection keypoint, M keypoint pairs are obtained, including:

[0136] The maximum positioning distance A is determined from the Q positioning distances corresponding to each configuration key point;

[0137] The matching degree between each detection key point and each configuration key point is calculated using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A.

[0138] For each configuration key point, the detection key point with the highest matching degree is determined from Q detection key points, and the configuration key point and the detection key point with the highest matching degree are determined as key point pairs to obtain M key point pairs.

[0139] In this embodiment, since the positioning distance between each configuration key point and each detection key point is calculated, each configuration key point will have Q positioning distances. The maximum positioning distance A is then determined from these Q positioning distances. Using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A as parameters, the matching degree between each detection key point and each configuration key point is calculated using the following formula:

[0140]

[0141] Wherein, detection_score represents the detection score of each detection keypoint, location_distance represents the positioning distance between each configured keypoint and each detection keypoint, max location_distance represents the maximum positioning distance A, α represents the weight of the detection score, and β represents the weight of the positioning distance.

[0142] like Figure 5 As shown, for each configured keypoint in the pre-configured information, the keypoint with the highest matching degree is selected from the detected keypoints in the image as the matched keypoint pair, thus obtaining M keypoint pairs. In this embodiment, when matching the detected keypoints with the configured keypoints, not only the positioning distance between the detected keypoints and the configured keypoints is considered, but also the detection score of the detected keypoints is included. Using the detection score, the positioning distance, and the maximum positioning distance among the Q positioning distances corresponding to each configured keypoint, the matching degree between each detected keypoint and each configured keypoint is calculated. The detected keypoint with the highest matching degree and the configured keypoint are selected as the keypoint pair. Since the detection score of the detected keypoints is considered during matching, it is beneficial to reduce the occurrence of mismatches, thereby helping to obtain M keypoint pairs with high accuracy and stability.

[0143] 303: Determine the matching score of each keypoint pair based on pre-configured information.

[0144] 304: Based on the matching scores of each keypoint pair, N keypoint pairs are determined from M keypoint pairs. The matching score of any keypoint pair among the N keypoint pairs is greater than or equal to a preset score threshold, and N is greater than or equal to a preset quantity threshold.

[0145] In this embodiment of the application, since the pre-configuration information also includes the position information of the target point of the target object in the object to be detected, such as the position information of the center point of the cable socket in telecommunications equipment, the matching score of each key point pair is determined based on the pre-configuration information, including:

[0146] Based on the location information of the target point and the location information of the configuration key points in each key point pair, the target distance between the configuration key point in each key point pair and the target point is calculated to obtain M target distances;

[0147] Determine the maximum target distance from M target distances;

[0148] The matching score of each keypoint pair is calculated by using the matching degree between the configured keypoint and the detected keypoint, the target distance between the configured keypoint and the target point, and the maximum target distance.

[0149] Specifically, for each of the M keypoint pairs, the target distance between the configured keypoint and the target point in each keypoint pair is calculated, resulting in M ​​target distances. For example, this target distance can be the distance between the coordinates of the configured keypoint and the target point in each keypoint pair, or the distance between the relative coordinates of the configured keypoint relative to the configured reference point and the relative coordinates of the target point relative to the configured reference point. Using the matching degree between the configured keypoint and the detected keypoint in each keypoint pair, the target distance between the configured keypoint and the target point in each keypoint pair, and the maximum target distance among the M target distances as parameters, the matching score of each keypoint pair in the M keypoint pairs is calculated using the following formula:

[0150]

[0151] Wherein, target_distance represents the target distance between the configured keypoint and the target point in each keypoint pair, max target_distance represents the maximum target distance among the M target distances, and γ represents the weight of the target distance.

[0152] Based on the matching scores of each of the M keypoint pairs, N keypoint pairs with matching scores greater than or equal to a preset score threshold are selected as the optimal keypoint pairs, such as... Figure 6 As shown, keypoint pairs consisting of two single screws and two double screws are selected as optimal keypoint pairs, respectively. Similarly, based on the need to calculate the mapping matrix, N must be greater than or equal to 4. In this embodiment, when calculating the matching score of each keypoint pair, in addition to the original matching degree of each keypoint pair (considering the detection score and positioning distance), the target distance, which can represent the positional relationship between the configured keypoint and the target point in the object to be detected, is also considered. The N keypoint pairs selected based on the detection score, positioning distance, and target distance are the optimal matching keypoint pairs. Using the N optimal matching keypoint pairs to map the target point helps to reduce the mapping deviation of the target point, thereby making the positional information of the target object in the image more accurate.

[0153] 305: Based on the position information of N key point pairs, obtain the position information of the target object in the object to be detected.

[0154] In this embodiment of the application, based on the location information of the configured key points and the location information of the detected key points in N key point pairs, a mapping matrix from the N configured key points in the N key point pairs to the N detected key points in the N key point pairs can be calculated, such as... Figure 7 As shown, this mapping matrix can be used to map target points from pre-configured information onto an image, thus obtaining the target object (e.g., ...). Figure 7 The location information of the cable connector (in the image) in the image. For example, this mapping matrix can be a perspective transformation matrix.

[0155] As can be seen, the embodiments of this application match the Q detection key points with the P configuration key points and the Q detection key points based on the location information of P configuration key points and the location information of Q detection key points. This can obtain the detection key points with the highest matching degree with the configuration key points. The resulting M key point pairs are all accurate and stable key point pairs. For the accurate and stable M key point pairs, the final matching score of each key point pair is calculated based on the pre-configured information. The key point pairs with matching scores greater than or equal to the preset threshold are selected. This will result in the N key point pairs with the best matching and the most favorable conditions for localization mapping. Based on the N key point pairs with the best matching and the most favorable conditions for localization mapping, the target object in the object to be detected is mapped to the corresponding image. This is beneficial for accurately and effectively detecting the position of the target object in the image. It is also applicable to target objects with occlusion and low resolution, thereby improving the effectiveness and accuracy of object detection.

[0156] The object detection method provided in this application can be applied to AI (Artificial Intelligence) intelligent operation and maintenance scenarios, accurately detecting difficult-to-detect components within objects and supporting component status monitoring. Compared to the computational load required by deep neural network object detection schemes, the computational load introduced by the location mapping-based scheme in this application is negligible, and the real-time performance of detection is relatively high. At the software and hardware architecture level, the object detection method provided in this application does not require software or hardware upgrades; it can be quickly adjusted simply by configuring the objects within different objects to be detected, resulting in a wide range of applications and high generalization ability.

[0157] In one application scenario of this application embodiment, the image of the object to be detected can be a video. For each frame of the video, the object detection method shown in steps 301-305 can be executed to locate the position of the target object in each frame, so as to achieve tracking of the target object. Furthermore, to ensure the stability of tracking, the detection results in each frame can be smoothed, specifically including detecting key points and reference points, so that the position of the target object in each frame is more stable and consistent. Furthermore, based on the M key point pairs selected in each frame, the configuration key points and detection key points in the optimal key point pair can be rewarded, and the configuration key points and detection key points in key point pairs with matching scores below a certain value can be penalized, so that the same optimal key point pair can be matched in the next frame of each frame, making the selection of key points more stable. In another application scenario of this application embodiment, when the image of the object to be detected is rotated, for example... Figure 8 The non-frontal image shown can be rotated inversely by the electronic device to obtain a frontal view image. Then, the object detection method shown in steps 301-305 is performed on the frontal view image to locate the position of the target object in the frontal view image. Then, the position of the target object in the original image is obtained through a rotation, so as to realize the localization and mapping of the target object that is difficult to detect in the rotating image of the object to be detected.

[0158] Please see Figure 9 , Figure 9 This is a flowchart illustrating another object detection method provided in an embodiment of this application. This method can also be applied to... Figure 2 In the scenario shown, the specific actions can be performed by electronic devices, such as... Figure 9 As shown, the method may include steps 901-907:

[0159] 901: Obtain the pre-configuration information of the object to be detected and obtain the detection results of the image of the object to be detected. The pre-configuration information includes the position information and category information of P configuration key points in the object to be detected. The detection results include the position information, detection score and category information of Q detection key points in the image.

[0160] In this embodiment, it should be understood that when configuring key points, reference points, and target points in the object to be detected, category information of these points is also configured. For example, the category information of P configured key points can be single screw, double screw, etc. Correspondingly, the detection result obtained by the object detection model from the image of the object to be detected includes not only the position information and detection score of key points and reference points in the image, but also the category information of key points and reference points in the image. For example, the category information of Q detected key points can be single screw, double screw, etc. Of course, the number of categories of Q detected key points may be more or less than the number of categories of P configured key points, but this does not limit the embodiments of this application.

[0161] 902: Based on the category information of each configuration key point and the category information of each detection key point, the configuration key points and detection key points of the same category among P configuration key points and Q detection key points are determined as key point sets to obtain at least one key point set of the same type.

[0162] In this embodiment, configuration key points and detection key points of the same category among P configuration key points and Q detection key points can be considered as a key point set to obtain at least one set of key points of the same category. The number of such sets is related to the total number of categories of the P configuration key points and Q detection key points. For example, the key points corresponding to a single screw in the pre-configuration information of a telecommunications device and a single screw in an image belong to the same set of key points; the key points corresponding to a double screw in the pre-configuration information of a telecommunications device and a double screw in an image belong to the same set of key points, and so on.

[0163] 903: For any set of key points H in at least one set of key points of the same type, based on the position information of R configuration key points and S detection key points in any set of key points H, obtain the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points.

[0164] In this embodiment of the application, R is greater than or equal to 1 and less than P; S is greater than or equal to 1 and less than Q, that is, in any set of key points of the same type H, there is at least one configuration key point and one detection key point.

[0165] For example, the pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and detection reference points correspond to the same position in the object to be detected, such as the lower left corner of the panel of a telecommunications device, based on the position information of R configuration key points and S detection key points in any set of key points of the same type H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained, including:

[0166] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0167] Based on the location information of the detection reference point and the location information of S detection key points, obtain the S fifth relative coordinates of the S detection key points relative to the detection reference point;

[0168] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S fifth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0169] Specifically, the relative coordinates of each configured keypoint B in any set of similar keypoints H relative to the configured reference point are called the fourth relative coordinates. Therefore, R configured keypoints will have R fourth relative coordinates. Similarly, the relative coordinates of each detected keypoint in any set of similar keypoints H relative to the detected reference point are called the fifth relative coordinates. Therefore, S detected keypoints will have S fifth relative coordinates. By calculating the Euclidean distance between each fourth relative coordinate and each fifth relative coordinate, the positioning distance between each configured keypoint B and each detected keypoint A is obtained. In this implementation, it should be understood that for any set of key points H of the same type, since only intra-category matching is required, when the configuration reference point and the detection reference point correspond to the same position in the object to be detected, calculating the positioning distance only requires calculating the R fourth relative coordinates of the R configuration key points relative to the configuration reference point, and only requires calculating the S fifth relative coordinates of the S detection key points relative to the detection reference point. Then, the Euclidean distance between each fourth relative coordinate and each fifth relative coordinate is calculated, thus obtaining the positioning distance between each detection key point A and each configuration key point B. Using the positioning distance between each detection key point A and each configuration key point B as a reference index for intra-category key point pair matching is beneficial for matching key point pairs that are more similar in layout position within the same category. Moreover, since the positioning distance is calculated only within the category, the amount of computation is greatly reduced.

[0170] For example, the pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and detection reference points correspond to different positions in the object to be detected, for example, the configuration reference point corresponds to the lower left corner of the panel of the telecommunications equipment, and the detection reference point corresponds to the upper right corner of the detection box of the panel of the telecommunications equipment, based on the position information of R configuration key points and S detection key points in any set of key points of the same type H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained, including:

[0171] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0172] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0173] Based on the location information of S detection key points, the location information of detection reference points, and the relative coordinates of detection reference points with respect to configuration reference points, obtain the S sixth relative coordinates of the S detection key points with respect to the configuration reference points;

[0174] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S sixth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0175] Specifically, for any set of key points H of the same type, when performing intra-class matching, it is necessary to obtain the relative coordinates of the detection reference point relative to the configuration reference point (or the relative coordinates of the configuration reference point relative to the detection reference point). Based on the position information of the detection key point, the position information of the detection reference point, and the relative coordinates of the detection reference point relative to the configuration reference point, the sixth relative coordinate of each of the S detection key points relative to the configuration reference point is calculated. Then, the Euclidean distance between the fourth and sixth relative coordinates is calculated to obtain the positioning distance between each detection key point A and each configuration key point B in any set of key points H of the same type. In this embodiment, when obtaining the positioning distance between each detection key point and each configuration key point in any set of key points H of the same type, the calculation of relative coordinates is transformed to a reference point at the same position in the device to be detected. This unifies the calculation standard of relative coordinates, which helps to avoid misjudging detection key points and configuration key points that are actually close in positioning distance as far in positioning distance, and vice versa, thereby reducing the mismatch rate between detection key points and configuration key points.

[0176] 904: Based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, K key point pairs are obtained. After processing at least one set of key points of the same type, M key point pairs are obtained, where the M key point pairs include the K key point pairs.

[0177] In this embodiment, K is greater than or equal to 0 and less than or equal to R, meaning that any set of key points H of the same type may not match any key point pairs. For example, based on the positioning distance between each detected key point A and each configured key point B, and the detection score of each detected key point A, K key point pairs are obtained, including:

[0178] The maximum positioning distance B is determined from the S positioning distances corresponding to each configuration key point B;

[0179] The matching degree between each detection key point A and each configuration key point B is calculated using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and the maximum positioning distance B.

[0180] For each configuration key point B, the detection key point A with the highest matching degree with configuration key point B is determined from S detection key points. The configuration key point B and the detection key point A with the highest matching degree are determined as key point pairs to obtain K key point pairs.

[0181] Specifically, since the positioning distance between each configuration key point B and each detection key point A is calculated, each configuration key point B will have S positioning distances. The maximum positioning distance B is determined from these S positioning distances. Using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and this maximum positioning distance B as parameters, the matching degree between each detection key point A and each configuration key point B is calculated. The calculation can be found in [reference needed]. Figure 3 The description in step 302 of the illustrated embodiment.

[0182] For each configured keypoint B in any set of keypoints of the same type H, the keypoint A with the highest matching degree is selected from S detected keypoints as the matching keypoint pair, resulting in K keypoint pairs. The same processing is performed on each keypoint set in at least one set of keypoints of the same type, resulting in M ​​keypoint pairs. In this embodiment, when matching detected keypoint A with configured keypoint B, not only the positioning distance between detected keypoint A and configured keypoint B is considered, but also the detection score of detected keypoint A is included. Using this detection score, the positioning distance, and the maximum positioning distance (i.e., the maximum positioning distance B) among the S positioning distances corresponding to each configured keypoint B, the matching degree between each detected keypoint A and each configured keypoint B is calculated. The keypoint A with the highest matching degree and the configured keypoint B are selected as the keypoint pair. Since the detection score of detected keypoint A is considered during matching, it helps reduce mismatches within the same category, thus helping to obtain K keypoint pairs with high accuracy and stability.

[0183] 905: Determine the matching score of each keypoint pair based on pre-configured information.

[0184] 906: Based on the matching scores of each keypoint pair, determine N keypoint pairs from M keypoint pairs. The matching score of any keypoint pair among the N keypoint pairs is greater than or equal to a preset score threshold.

[0185] 907: Based on the position information of N key point pairs, obtain the position information of the target object in the object to be detected.

[0186] In this embodiment of the application, N is greater than or equal to a preset quantity threshold, wherein the specific implementation of steps 905-907 is as follows: Figure 3 The embodiments shown have been described in detail and can achieve the same or similar beneficial effects, so they will not be repeated here.

[0187] As can be seen, in the embodiments of this application, when matching Q detection key points with P configuration key points, the P configuration key points and Q detection key points can be classified according to the category information of each configuration key point and each detection key point to obtain at least one set of key points of the same type. Based on obtaining at least one set of key points of the same type, for any set of key points of the same type H, it is only necessary to match S detection key points with R configuration key points in any set of key points of the same type H based on the positioning distance between each detection key point A and each configuration key point B and the detection score of each detection key point A to obtain K key point pairs, without the need for inter-category matching. This not only solves the mismatch problem caused by matching based solely on positioning distance and matches M accurate and stable key point pairs, but also relatively reduces the amount of computation and matching.

[0188] The methods provided in the embodiments of this application have been described in detail above. The apparatus and devices of the embodiments of this application are provided below.

[0189] Please see Figure 10 , Figure 10 This is a schematic diagram of an object detection device provided in an embodiment of this application. The object detection device 100 is applied to the aforementioned electronic device, such as... Figure 10 As shown, the device includes an acquisition unit 1001 and a processing unit 1002; wherein, the acquisition unit is used to acquire pre-configuration information of the object to be detected and to acquire detection results of the image of the object to be detected. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection results include the position information of Q detection key points in the image. P and Q are both greater than or equal to a preset number threshold.

[0190] The processing unit is used to obtain M key point pairs based on the location information of P configuration key points and the location information of Q detection key points. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to a preset number threshold.

[0191] The processing unit is also used to determine the matching score of each key point pair based on pre-configured information;

[0192] The processing unit is also used to determine N key point pairs from M key point pairs based on the matching scores of each key point pair, wherein the matching score of any key point pair among the N key point pairs is greater than or equal to a preset score threshold, and N is greater than or equal to a preset quantity threshold.

[0193] The processing unit is also used to obtain the position information of the target object in the object to be detected based on the position information of N key point pairs.

[0194] It can be seen that, Figure 10In the device shown, based on the position information of P configuration key points and Q detection key points, the Q detection key points are matched with the P configuration key points to obtain the detection key points with the highest matching degree. The resulting M key point pairs are all accurate and stable key point pairs. For the accurate and stable M key point pairs, the final matching score of each key point pair is calculated based on the pre-configured information. The key point pairs with matching scores greater than or equal to a preset threshold are selected. In this way, the N key point pairs with the best matching and most conducive to localization mapping are obtained. Based on the N key point pairs with the best matching and most conducive to localization mapping, the target object in the object to be detected is mapped to the corresponding image. This is beneficial for accurately and effectively detecting the position of the target object in the image and is applicable to target objects with occlusion and low resolution, thereby improving the effectiveness and accuracy of object detection.

[0195] In one possible implementation, the detection result also includes the detection score of each of the Q detection key points. Regarding obtaining M key point pairs based on the location information of P configuration key points and the location information of the Q detection key points, the processing unit 1002 is specifically used for:

[0196] Based on the location information of P configuration key points and Q detection key points, the positioning distance between each detection key point and each configuration key point among the P configuration key points is obtained.

[0197] Based on the positioning distance between each detection key point and each configuration key point, and the detection score of each detection key point, M key point pairs are obtained.

[0198] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to the same position in the object to be detected, the processing unit 1002 is specifically used to: obtain the positioning distance between each detection key point and each of the P configuration key points based on the position information of P configuration key points and the position information of Q detection key points.

[0199] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0200] Based on the location information of the detection reference point and the location information of Q detection key points, obtain the Q second relative coordinates of the Q detection key points relative to the detection reference point;

[0201] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q second relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0202] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to different positions in the object to be detected, in terms of obtaining the positioning distance between each detection key point and each of the P configuration key points based on the position information of P configuration key points and the position information of Q detection key points, the processing unit 1002 is specifically used for:

[0203] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0204] Based on the location information of the configuration reference point and the location information of P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point;

[0205] Based on the location information of Q key detection points, the location information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configured reference point, obtain the Q third relative coordinates of the Q key detection points with respect to the configured reference point;

[0206] Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q third relative coordinates, and determine the Euclidean distance between each detection key point and each configuration key point.

[0207] In one possible implementation, regarding obtaining M keypoint pairs based on the positioning distance between each detected keypoint and each configured keypoint, and the detection score of each detected keypoint, the processing unit 1002 is specifically used for:

[0208] The maximum positioning distance A is determined from the Q positioning distances corresponding to each configuration key point;

[0209] The matching degree between each detection key point and each configuration key point is calculated using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A.

[0210] For each configuration key point, the detection key point with the highest matching degree is determined from Q detection key points, and the configuration key point and the detection key point with the highest matching degree are determined as key point pairs to obtain M key point pairs.

[0211] In one possible implementation, the pre-configuration information further includes category information for each of the P configuration key points, and the detection result further includes the detection score and category information for each of the Q detection key points. Regarding obtaining M key point pairs based on the location information of the P configuration key points and the location information of the Q detection key points, the processing unit 1002 is specifically used for:

[0212] Based on the category information of each configuration key point and the category information of each detection key point, the configuration key points and detection key points of the same category among the P configuration key points and Q detection key points are determined as key point sets, so as to obtain at least one key point set of the same type;

[0213] For any set of key points H in at least one set of key points of the same type, based on the location information of R configuration key points and S detection key points in any set of key points H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained; R is greater than or equal to 1 and less than P; S is greater than or equal to 1 and less than Q.

[0214] Based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, K key point pairs are obtained. M key point pairs include K key point pairs, where K is greater than or equal to 0 and less than or equal to R.

[0215] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to the same position in the object to be detected, the processing unit 1002 is specifically used to: obtain the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points based on the position information of R configuration key points and S detection key points in any set of key points of the same type H.

[0216] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0217] Based on the location information of the detection reference point and the location information of S detection key points, obtain the S fifth relative coordinates of the S detection key points relative to the detection reference point;

[0218] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S fifth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0219] In one possible implementation, the pre-configuration information further includes the position information of configuration reference points in the object to be detected, and the detection result further includes the position information of detection reference points in the image. When the configuration reference points and detection reference points correspond to different positions in the object to be detected, in terms of obtaining the positioning distance between each detection key point A among the S detection key points and each configuration key point B among the R configuration key points based on the position information of R configuration key points and S detection key points in any set of key points of the same type H, the processing unit 1002 is specifically used for:

[0220] Based on the location information of the configuration reference point and the location information of the detection reference point, obtain the relative coordinates of the detection reference point with respect to the configuration reference point;

[0221] Based on the location information of the configuration reference point and the location information of R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point;

[0222] Based on the location information of S detection key points, the location information of detection reference points, and the relative coordinates of detection reference points with respect to configuration reference points, obtain the S sixth relative coordinates of the S detection key points with respect to the configuration reference points;

[0223] Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S sixth relative coordinates, and determine the Euclidean distance between each detection key point A and each configuration key point B.

[0224] In one possible implementation, regarding obtaining K keypoint pairs based on the positioning distance between each detection keypoint A and each configuration keypoint B, and the detection score of each detection keypoint A, the processing unit 1002 is specifically used for:

[0225] The maximum positioning distance B is determined from the S positioning distances corresponding to each configuration key point B;

[0226] The matching degree between each detection key point A and each configuration key point B is calculated using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and the maximum positioning distance B.

[0227] For each configuration key point B, the detection key point A with the highest matching degree with configuration key point B is determined from S detection key points. The configuration key point B and the detection key point A with the highest matching degree are determined as key point pairs to obtain K key point pairs.

[0228] In one possible implementation, the pre-configuration information also includes the position information of the target point of the target object in the object to be detected. Specifically, in determining the matching score of each keypoint pair based on the pre-configuration information, the processing unit 1002 is used for:

[0229] Based on the location information of the target point and the location information of the configuration key points in each key point pair, the target distance between the configuration key point in each key point pair and the target point is calculated to obtain M target distances;

[0230] Determine the maximum target distance from M target distances;

[0231] The matching score of each keypoint pair is calculated by using the matching degree between the configured keypoint and the detected keypoint, the target distance between the configured keypoint and the target point, and the maximum target distance.

[0232] In one possible implementation, the pre-configuration information also includes the position information of the target point of the target object in the object to be detected. Specifically, the processing unit 1002 is used to obtain the position information of the target object in the object to be detected based on the position information of N key point pairs:

[0233] Based on the location information of N key point pairs, a mapping matrix is ​​calculated from the N configured key points in the N key point pairs to the N detected key points in the N key point pairs;

[0234] The target points are mapped onto the image using a mapping matrix to obtain the location information of the target object.

[0235] According to one embodiment of this application, Figure 10 The various units in the object detection device 100 shown can be individually or entirely combined into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the object detection device 1700 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0236] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), capable of performing operations such as... Figure 3 or Figure 9 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 10 The object detection apparatus 100 shown herein, and the object detection method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0237] Based on the description of the method and device embodiments above, this application also provides an electronic device. Please refer to... Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 110 includes at least a processor 1101, a memory 1102, an input device 1103, and an output device 1104, as well as one or more programs. The one or more programs are stored in the memory 1102. The various parts are interconnected via a bus 1105 or other means. The input device 1103 and the output device 1104 can be the same device. The input device 1103 can be used to receive input from other devices or developers, and the output device 1104 can be used to output object detection results. Figure 11 as well as Figure 11 The various units shown can be implemented using the processor 1101, memory 1102, input device 1103, and output device 1104 described above.

[0238] The memory 1102 includes, but is not limited to, RAM, ROM, erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used to store related computer programs and data.

[0239] Processor 1101 can be one or more CPUs. When processor 1101 is a CPU, the CPU can be a single-core CPU or a multi-core CPU.

[0240] The processor 1101 in the electronic device 110 is used to read one or more programs stored in the memory 1102 and perform the following operations:

[0241] The pre-configuration information of the object to be detected and the detection result of the image of the object to be detected are obtained. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection result includes the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold.

[0242] Based on the location information of P configuration key points and Q detection key points, M key point pairs are obtained. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to a preset number threshold.

[0243] The matching score of each key point pair is determined based on pre-configured information;

[0244] Based on the matching scores of each keypoint pair, N keypoint pairs are determined from M keypoint pairs. The matching score of any keypoint pair among the N keypoint pairs is greater than or equal to a preset score threshold, and N is greater than or equal to a preset quantity threshold.

[0245] Based on the position information of N key point pairs, the position information of the target object in the object to be detected is obtained.

[0246] It can be seen that, Figure 11 In the electronic device 110 shown, based on the position information of P configuration key points and the position information of Q detection key points, the Q detection key points are matched with the P configuration key points to obtain the detection key points with the highest matching degree. The resulting M key point pairs are all accurate and stable key point pairs. For the accurate and stable M key point pairs, the final matching score of each key point pair is calculated based on the pre-configured information. The key point pairs with matching scores greater than or equal to the preset threshold are selected. In this way, the N key point pairs with the best matching and most conducive to localization mapping are obtained. Based on the N key point pairs with the best matching and most conducive to localization mapping, the target object in the object to be detected is mapped to the corresponding image. This is beneficial for accurately and effectively detecting the position of the target object in the image and is applicable to target objects with occlusion and low resolution, thereby improving the effectiveness and accuracy of object detection.

[0247] It should be noted that the implementation of each operation can also be referred to accordingly. Figure 3 The corresponding description of the method embodiments shown.

[0248] It should be noted that, although Figure 11The illustrated electronic device 110 only shows a processor 1101, a memory 1102, an input device 1103, an output device 1104, and a bus 1105. However, in specific implementations, those skilled in the art should understand that the electronic device 110 also includes other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the electronic device 110 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the electronic device 110 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 11 All the devices shown.

[0249] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in the electronic device 110 for storing a computer program for execution by the device. When the program is run on the electronic device 110, Figure 3 or Figure 9 The illustrated method flow is thus implemented. It is understood that the computer-readable storage medium here may include both the built-in storage medium in electronic device 110 and extended storage media supported by electronic device 110. The computer-readable storage medium provides storage space containing the operating system of electronic device 110. Furthermore, one or more computer programs suitable for loading and execution by processor 1101 are also stored in this storage space. It should be noted that the computer-readable storage medium here may be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 1101.

[0250] This application also provides a computer program product that, when run by an electronic device, provides a solution for... Figure 3 or Figure 9 The method flow shown is thus implemented.

[0251] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0252] It should be understood that the processor mentioned in the embodiments of this application can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0253] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be ROM, Programmable Read-Only Memory (PROM), EPROM, Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory. Volatile memory can be RAM, which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchlink Dynamic Random Access Memory (SLDRAM), and Direct Rambus RAM (DRRAM).

[0254] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor.

[0255] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0256] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0257] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely exemplary. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0258] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0259] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0260] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0261] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0262] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0263] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An object detection method, characterized in that, The method includes: The pre-configuration information of the object to be detected and the detection result of the image of the object to be detected are obtained. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection result includes the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold. Based on the location information of the P configuration key points and the location information of the Q detection key points, M key point pairs are obtained. Each key point pair in the M key point pairs includes one configuration key point from the P configuration key points and one detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to the preset number threshold. The matching score of each key point pair is determined based on the pre-configured information; Based on the matching scores of each key point pair, N key point pairs are determined from the M key point pairs. The matching score of any key point pair in the N key point pairs is greater than or equal to a preset score threshold, and N is greater than or equal to the preset quantity threshold. Based on the position information of the N key point pairs, the position information of the target object in the object to be detected is obtained.

2. The method according to claim 1, characterized in that, The detection results also include the detection score of each of the Q detection key points. Based on the location information of the P configuration key points and the location information of the Q detection key points, M key point pairs are obtained, including: Based on the location information of the P configuration key points and the location information of the Q detection key points, the positioning distance between each detection key point and each configuration key point among the P configuration key points is obtained; Based on the positioning distance between each detection key point and each configuration key point, and the detection score of each detection key point, the M key point pairs are obtained.

3. The method according to claim 2, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to the same position in the object to be detected, the step of obtaining the positioning distance between each detection key point and each of the P configuration key points based on the position information of the P configuration key points and the position information of the Q detection key points includes: Based on the location information of the configuration reference point and the location information of the P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point; Based on the location information of the detection reference point and the location information of the Q key detection points, obtain the Q second relative coordinates of the Q key detection points relative to the detection reference point; Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q second relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point and each configuration key point.

4. The method according to claim 2, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to different positions in the object to be detected, the step of obtaining the positioning distance between each detection key point and each of the P configuration key points based on the position information of the P configuration key points and the position information of the Q detection key points includes: Based on the location information of the configuration reference point and the location information of the detection reference point, the relative coordinates of the detection reference point with respect to the configuration reference point are obtained; Based on the location information of the configuration reference point and the location information of the P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point; Based on the position information of the Q detection key points, the position information of the detection reference point, and the relative coordinates of the detection reference point relative to the configuration reference point, obtain the Q third relative coordinates of the Q detection key points relative to the configuration reference point; Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q third relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point and each configuration key point.

5. The method according to claim 3 or 4, characterized in that, The M keypoint pairs are obtained based on the positioning distance between each detected keypoint and each configured keypoint, and the detection score of each detected keypoint, including: The maximum positioning distance A is determined from the Q positioning distances corresponding to each configuration key point; The matching degree between each detection key point and each configuration key point is calculated using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A. For each configuration key point, the detection key point with the highest matching degree with the configuration key point is determined from the Q detection key points, and the configuration key point with the highest matching degree with the configuration key point is determined as a key point pair to obtain the M key point pairs.

6. The method according to claim 1, characterized in that, The pre-configuration information also includes category information for each of the P configuration key points, and the detection result also includes the detection score and category information for each of the Q detection key points. Based on the location information of the P configuration key points and the location information of the Q detection key points, M key point pairs are obtained, including: Based on the category information of each configuration key point and the category information of each detection key point, configuration key points and detection key points of the same category among the P configuration key points and the Q detection key points are determined as key point sets, so as to obtain at least one key point set of the same type. For any one set of key points H in the at least one set of key points of the same type, based on the position information of R configuration key points and S detection key points in the set of key points H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained; R is greater than or equal to 1 and less than P; S is greater than or equal to 1 and less than Q. Based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, K key point pairs are obtained. The M key point pairs include the K key point pairs, where K is greater than or equal to 0 and less than or equal to R.

7. The method according to claim 6, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to the same position in the object to be detected, the step of obtaining the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points based on the position information of R configuration key points and S detection key points in any set of similar key points H includes: Based on the location information of the configuration reference point and the location information of the R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point; Based on the location information of the detection reference point and the location information of the S key detection points, obtain the S fifth relative coordinates of the S key detection points relative to the detection reference point; Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S fifth relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point A and each configuration key point B.

8. The method according to claim 6, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to different positions in the object to be detected, the step of obtaining the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points based on the position information of R configuration key points and S detection key points in any set of key points of the same type H includes: Based on the location information of the configuration reference point and the location information of the detection reference point, the relative coordinates of the detection reference point with respect to the configuration reference point are obtained; Based on the location information of the configuration reference point and the location information of the R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point; Based on the location information of the S detection key points, the location information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configuration reference point, obtain the S sixth relative coordinates of the S detection key points with respect to the configuration reference point; Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S sixth relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point A and each configuration key point B.

9. The method according to claim 7 or 8, characterized in that, Based on the positioning distance between each detection key point A and each configured key point B, and the detection score of each detection key point A, K key point pairs are obtained, including: The maximum positioning distance B is determined from the S positioning distances corresponding to each configuration key point B; The matching degree between each detection key point A and each configuration key point B is calculated using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and the maximum positioning distance B. For each configuration key point B, the detection key point A with the highest matching degree with the configuration key point B is determined from the S detection key points, and the configuration key point B and the detection key point A with the highest matching degree are determined as key point pairs to obtain the K key point pairs.

10. The method according to claim 5 or 9, characterized in that, The pre-configuration information also includes the position information of the target point of the target object in the object to be detected, and the step of determining the matching score of each key point pair based on the pre-configuration information includes: Based on the location information of the target point and the location information of the configuration key points in each key point pair, the target distance between the configuration key points in each key point pair and the target point is calculated to obtain M target distances; Determine the maximum target distance from the M target distances; The matching score of each key point pair is calculated using the matching degree between the configured key point and the detected key point, the target distance between the configured key point and the target point, and the maximum target distance.

11. The method according to any one of claims 1-10, characterized in that, The pre-configuration information also includes the position information of the target point of the target object in the object to be detected. The step of obtaining the position information of the target object in the object to be detected based on the position information of the N key point pairs includes: Based on the location information of the N key point pairs, a mapping matrix is ​​calculated from the N configured key points in the N key point pairs to the N detected key points in the N key point pairs; The target point is mapped onto the image using the mapping matrix to obtain the location information of the target object.

12. An object detection device, characterized in that, The device includes an acquisition unit and a processing unit, wherein... The acquisition unit is used to acquire pre-configuration information of the object to be detected and to acquire detection results of the image of the object to be detected. The pre-configuration information includes the position information of P configuration key points in the object to be detected, and the detection results include the position information of Q detection key points in the image. Both P and Q are greater than or equal to a preset number threshold. The processing unit is used to obtain M key point pairs based on the location information of the P configuration key points and the location information of the Q detection key points. Each key point pair in the M key point pairs includes a configuration key point from the P configuration key points and a detection key point from the Q detection key points that matches the configuration key point. M is greater than or equal to the preset number threshold. The processing unit is further configured to determine the matching score of each key point pair based on the pre-configured information; The processing unit is further configured to determine N key point pairs from the M key point pairs based on the matching scores of each key point pair, wherein the matching score of any key point pair among the N key point pairs is greater than or equal to a preset score threshold, and N is greater than or equal to the preset quantity threshold. The processing unit is further configured to obtain the position information of the target object in the object to be detected based on the position information of the N key point pairs.

13. The apparatus according to claim 12, characterized in that, The detection result also includes the detection score of each of the Q detection key points. Regarding obtaining M key point pairs based on the location information of the P configuration key points and the location information of the Q detection key points, the processing unit is specifically used for: Based on the location information of the P configuration key points and the location information of the Q detection key points, the positioning distance between each detection key point and each configuration key point among the P configuration key points is obtained; Based on the positioning distance between each detection key point and each configuration key point, and the detection score of each detection key point, the M key point pairs are obtained.

14. The apparatus according to claim 13, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to the same position in the object to be detected, in terms of obtaining the positioning distance between each detection key point and each of the P configuration key points based on the position information of the P configuration key points and the position information of the Q detection key points, the processing unit is specifically used for: Based on the location information of the configuration reference point and the location information of the P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point; Based on the location information of the detection reference point and the location information of the Q key detection points, obtain the Q second relative coordinates of the Q key detection points relative to the detection reference point; Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q second relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point and each configuration key point.

15. The apparatus according to claim 13, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to different positions in the object to be detected, in terms of obtaining the positioning distance between each detection key point and each of the P configuration key points based on the position information of the P configuration key points and the position information of the Q detection key points, the processing unit is specifically used for: Based on the location information of the configuration reference point and the location information of the detection reference point, the relative coordinates of the detection reference point with respect to the configuration reference point are obtained; Based on the location information of the configuration reference point and the location information of the P configuration key points, obtain the P first relative coordinates of the P configuration key points relative to the configuration reference point; Based on the position information of the Q detection key points, the position information of the detection reference point, and the relative coordinates of the detection reference point relative to the configuration reference point, obtain the Q third relative coordinates of the Q detection key points relative to the configuration reference point; Calculate the Euclidean distance between each of the P first relative coordinates and each of the Q third relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point and each configuration key point.

16. The apparatus according to claim 14 or 15, characterized in that, In obtaining the M keypoint pairs based on the positioning distance between each detected keypoint and each configured keypoint, and the detection score of each detected keypoint, the processing unit is specifically used for: The maximum positioning distance A is determined from the Q positioning distances corresponding to each configuration key point; The matching degree between each detection key point and each configuration key point is calculated using the detection score of each detection key point, the positioning distance between each detection key point and each configuration key point, and the maximum positioning distance A. For each configuration key point, the detection key point with the highest matching degree with the configuration key point is determined from the Q detection key points, and the configuration key point with the highest matching degree with the configuration key point is determined as a key point pair to obtain the M key point pairs.

17. The apparatus according to claim 12, characterized in that, The pre-configuration information further includes category information for each of the P configuration key points, and the detection result further includes the detection score and category information for each of the Q detection key points. Regarding obtaining M key point pairs based on the location information of the P configuration key points and the location information of the Q detection key points, the processing unit is specifically used for: Based on the category information of each configuration key point and the category information of each detection key point, configuration key points and detection key points of the same category among the P configuration key points and the Q detection key points are determined as key point sets, so as to obtain at least one key point set of the same type. For any one set of key points H in the at least one set of key points of the same type, based on the position information of R configuration key points and S detection key points in the set of key points H, the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points is obtained; R is greater than or equal to 1 and less than P; S is greater than or equal to 1 and less than Q. Based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, K key point pairs are obtained. The M key point pairs include the K key point pairs, where K is greater than or equal to 0 and less than or equal to R.

18. The apparatus according to claim 17, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to the same position in the object to be detected, the processing unit is specifically used to obtain the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points based on the position information of R configuration key points and S detection key points in any set of similar key points H. Based on the location information of the configuration reference point and the location information of the R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point; Based on the location information of the detection reference point and the location information of the S key detection points, obtain the S fifth relative coordinates of the S key detection points relative to the detection reference point; Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S fifth relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point A and each configuration key point B.

19. The apparatus according to claim 17, characterized in that, The pre-configuration information also includes the position information of the configuration reference points in the object to be detected, and the detection result also includes the position information of the detection reference points in the image. When the configuration reference points and the detection reference points correspond to different positions in the object to be detected, in terms of obtaining the positioning distance between each detection key point A in the S detection key points and each configuration key point B in the R configuration key points based on the position information of R configuration key points and S detection key points in any set of similar key points H, the processing unit is specifically used for: Based on the location information of the configuration reference point and the location information of the detection reference point, the relative coordinates of the detection reference point with respect to the configuration reference point are obtained; Based on the location information of the configuration reference point and the location information of the R configuration key points, obtain the R fourth relative coordinates of the R configuration key points relative to the configuration reference point; Based on the location information of the S detection key points, the location information of the detection reference point, and the relative coordinates of the detection reference point with respect to the configuration reference point, obtain the S sixth relative coordinates of the S detection key points with respect to the configuration reference point; Calculate the Euclidean distance between each of the R fourth relative coordinates and each of the S sixth relative coordinates, and determine the Euclidean distance as the positioning distance between each detection key point A and each configuration key point B.

20. The apparatus according to claim 18 or 19, characterized in that, In obtaining K key point pairs based on the positioning distance between each detection key point A and each configuration key point B, and the detection score of each detection key point A, the processing unit is specifically used for: The maximum positioning distance B is determined from the S positioning distances corresponding to each configuration key point B; The matching degree between each detection key point A and each configuration key point B is calculated using the detection score of each detection key point A, the positioning distance between each detection key point A and each configuration key point B, and the maximum positioning distance B. For each configuration key point B, the detection key point A with the highest matching degree with the configuration key point B is determined from the S detection key points, and the configuration key point B and the detection key point A with the highest matching degree are determined as key point pairs to obtain the K key point pairs.

21. The apparatus according to claim 16 or 20, characterized in that, The pre-configuration information also includes the position information of the target point of the target object in the object to be detected. In determining the matching score of each keypoint pair based on the pre-configuration information, the processing unit is specifically used for: Based on the location information of the target point and the location information of the configuration key points in each key point pair, the target distance between the configuration key points in each key point pair and the target point is calculated to obtain M target distances; Determine the maximum target distance from the M target distances; The matching score of each key point pair is calculated using the matching degree between the configured key point and the detected key point, the target distance between the configured key point and the target point, and the maximum target distance.

22. The apparatus according to any one of claims 12-21, characterized in that, The pre-configuration information also includes the position information of the target point of the target object in the object to be detected. Specifically, in obtaining the position information of the target object in the object to be detected based on the position information of the N key point pairs, the processing unit is used for: Based on the location information of the N key point pairs, a mapping matrix is ​​calculated from the N configured key points in the N key point pairs to the N detected key points in the N key point pairs; The target point is mapped onto the image using the mapping matrix to obtain the location information of the target object.

23. An electronic device, characterized in that, It includes a processor, a memory, and one or more programs, the processor being connected to the memory, the one or more programs being stored in the memory, and being configured to implement the method as described in any one of claims 1-11 when executed by the processor.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program for execution by the device, which, when executed, implements the method of any one of claims 1-11.

25. A computer program product, characterized in that, The computer program product includes one or more programs that, when run by an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

Images