Model detection result matching method and device, equipment and medium

Abstract

The embodiment of the invention provides a model detection result matching method and device, equipment and a medium, and relates to the technical field of image detection. The method comprises the steps of obtaining a first detection result set output by a target detection model and a second detection result set output by a key point detection model based on a to-be-detected image; extracting position change information corresponding to a first detection result set and a second detection result set between adjacent frames in the to-be-detected image; based on the first motion vector set, constructing a membership function for evaluating the motion consistency of the detection result; and inputting the second motion vector set into a membership function, and matching the first detection result set with the second detection result set based on an evaluation result output by the membership function. The method provided by the invention effectively solves the problem that the detection results of different models are difficult to match with each other in the prior art, so that the production safety detection effectiveness is influenced.

Description

Model detection result matching methods, devices, equipment and media Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a method, apparatus, device and medium for matching model detection results. Background Technology

[0002] In modern refining and chemical enterprises' safety production management, intelligent video monitoring systems have become a core tool. These systems typically integrate multiple computer vision models, such as simultaneously deploying a helmet detection model and a human body key point detection model. This multi-model collaborative working mode aims to achieve precise control from "whether a helmet is worn" to "which specific person is not wearing one," which is crucial for risk warning, violation tracing, and personnel behavior analysis.

[0003] In related technologies, each model is often deployed and run independently. That is, it can only output two types of results: "a safety helmet was detected" and "a human body and its key points were detected". There is no effective identity association mechanism between the two, and it is impossible to directly determine which set of key points belongs to which safety helmet. Moreover, due to the complexity of the on-site environment, the monitored objects are usually in constant dynamic change and interact with various objects on the site, making it difficult to effectively match the detection results of different models, thus affecting the effectiveness of production safety detection. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and medium for matching model detection results, in order to solve the problem in related technologies where the detection results of different models are difficult to match with each other, thereby affecting the effectiveness of production safety detection.

[0005] Firstly, this application provides a model detection result matching method, the method comprising the following steps:

[0006] Based on the image to be detected, a first set of detection results output by the target detection model and a second set of detection results output by the key point detection model are obtained, wherein the image to be detected includes multiple consecutive frames of images;

[0007] Extract position change information between adjacent frames in the image to be detected, corresponding to the first detection result set and the second detection result set. The position change information includes the first motion vector set corresponding to the first detection result set and the second motion vector set corresponding to the second detection result set.

[0008] Based on the first set of motion vectors, a membership function is constructed to evaluate the motion consistency of the detection results;

[0009] The second set of motion vectors is input into the membership function, and the first set of detection results and the second set of detection results are matched based on the evaluation results output by the membership function.

[0010] In one embodiment of this disclosure, the first detection result set includes a first detection result corresponding to each frame of the image to be detected, and the second detection result set includes a second detection result corresponding to each frame of the image to be detected. After obtaining the first detection result set output by the target detection model and the second detection result set output by the key point detection model based on the image to be detected, the method further includes: determining the distance between a first position in the first detection result and a second position in the second detection result based on an initial frame in the image to be detected; and determining an initial matching result between the first detection result and the second detection result based on the minimum distance principle.

[0011] In one embodiment of this disclosure, extracting positional change information corresponding to a first detection result set and a second detection result set between adjacent frames in an image to be detected includes: taking multiple consecutive frames as an inference unit and sequentially determining each pair of adjacent frames in the inference unit; for each pair of adjacent frames, based on the initial matching result, extracting the motion vectors of the matching first detection result and second detection result between the pair of adjacent frames, wherein the motion vector of the first detection result in the adjacent frame is the first motion vector, and the motion vector of the second detection result in the adjacent frame is the second motion vector; and obtaining a first motion vector set and a second motion vector set based on the motion vectors extracted from all adjacent frames in the inference unit.

[0012] In one embodiment of this disclosure, the membership function is a triangular membership function. Based on a first set of motion vectors, a membership function for evaluating the motion consistency of detection results is constructed, including: determining the magnitude of the first motion vector of each first detection result within the inference unit; dynamically determining the parameters of a triangular membership function based on the magnitude; and constructing the triangular membership function based on the parameters, wherein the triangular membership function is used to characterize the degree to which a given magnitude belongs to the normal motion range defined by the motion pattern of the first detection result.

[0013] In one embodiment of this disclosure, matching the first detection result set and the second detection result set based on the evaluation result output by the membership function includes: for each initial matching result, inputting the magnitude of its second motion vector in the inference unit into the membership function, and outputting the corresponding membership value; determining the proportion of membership values ​​greater than a preset first threshold to all membership values; if the proportion is greater than a preset second threshold, determining that the current initial matching result is valid and maintaining the matching relationship; otherwise, determining that the current initial matching result is a matching failure.

[0014] In one embodiment of this disclosure, after determining that the current initial matching result is a failed match, the method further includes: based on the current frame in the image to be detected, redetermining the distance between the second position of the failed second detection result and the first position of all first detection results; based on the minimum distance principle, reassigning the matched first detection results to the failed second detection result, and redetermining the corresponding evaluation result.

[0015] In one embodiment of this disclosure, the second position in the second detection result is a set position, which is obtained by: determining the extreme points of at least two key points contained in the second detection result in the image coordinate system; and calculating the set position representing the spatial range of the second detection result based on the coordinates of the extreme points.

[0016] Secondly, embodiments of this disclosure provide a model detection result matching device, which includes:

[0017] The determination module is used to obtain a first set of detection results output by the target detection model and a second set of detection results output by the key point detection model based on the image to be detected, wherein the image to be detected includes multiple consecutive frames of images;

[0018] The extraction module is used to extract position change information between adjacent frames in the image to be detected, corresponding to the first detection result set and the second detection result set. The position change information includes the first motion vector set corresponding to the first detection result set and the second motion vector set corresponding to the second detection result set.

[0019] A building module is used to construct a membership function for evaluating the motion consistency of the detection results based on the first set of motion vectors;

[0020] The processing module is used to input the second set of motion vectors into the membership function and match the first detection result set with the second detection result set based on the evaluation result output by the membership function.

[0021] Optionally, the determining module is specifically used to: if the first detection result set includes a first detection result corresponding to each frame of the image to be detected, and the second detection result set includes a second detection result corresponding to each frame of the image to be detected, then after obtaining the first detection result set and the second detection result set, based on the initial frame in the image to be detected, determine the distance between the first position in the first detection result and the second position in the second detection result, and based on the minimum distance principle, determine the initial matching result between the first detection result and the second detection result.

[0022] Optionally, the extraction module is specifically used to treat multiple consecutive frames of images as an inference unit, and sequentially determine each pair of adjacent frames in the inference unit; for each pair of adjacent frames, based on the initial matching result, extract the motion vectors of the matching first detection result and second detection result between the pair of adjacent frames, wherein the motion vector of the first detection result in the adjacent frame is the first motion vector, and the motion vector of the second detection result in the adjacent frame is the second motion vector; based on the motion vectors extracted from all adjacent frames in the inference unit, obtain the first motion vector set and the second motion vector set.

[0023] Optionally, the construction module is specifically used to: if the membership function is a triangular membership function, determine the magnitude of the first motion vector of each first detection result within the inference unit; dynamically determine the parameters of a triangular membership function based on the magnitude; and construct a triangular membership function based on the parameters, wherein the membership function is used to characterize the degree to which a given magnitude belongs to the normal motion range defined by the motion pattern of the first detection result.

[0024] Optionally, the processing module is specifically used to: for each initial matching result, input the magnitude of its second motion vector within the inference unit into the membership function and output the corresponding membership value; determine the proportion of membership values ​​greater than a preset first threshold to all membership values; if the proportion is greater than a preset second threshold, determine that the current initial matching result is valid and maintain the matching relationship; otherwise, determine that the current initial matching result is a matching failure.

[0025] Optionally, the processing module is further configured to, after determining that the current initial matching result is a failed match, redetermine the distance between the second position of the failed second detection result and the first position of all first detection results based on the current frame in the image to be detected; and, based on the minimum distance principle, reassign the matched first detection results to the failed second detection results and redetermine the corresponding evaluation results.

[0026] Optionally, the determining module is further configured to, if the second position in the second detection result is a set position, obtain the set position based on the following method: determine the extreme points of at least two key points contained in the second detection result in the image coordinate system; and calculate the set position representing the spatial range of the second detection result based on the coordinates of the extreme points.

[0027] Thirdly, embodiments of this application provide a control device, including: a memory and a processor;

[0028] The memory stores the instructions that the computer executes;

[0029] The processor executes computer execution instructions stored in memory, causing the processor to perform a model detection result matching method for implementing the first aspect of this disclosure.

[0030] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the model detection result matching method as described in the first aspect of this disclosure.

[0031] Fifthly, embodiments of this disclosure also provide a computer program product comprising computer execution instructions, which, when executed by a processor, are used to implement the model detection result matching method as described in the first aspect of this disclosure.

[0032] The model detection result matching method, apparatus, device, and medium provided in this disclosure acquire the detection results of different models in multiple consecutive frames of images, extract their positional change information between adjacent frames to obtain a set of motion vectors, construct a membership function to evaluate motion consistency based on this, and finally complete the matching based on the evaluation result output by the function. This effectively solves the problem of difficulty in matching detection results in multi-model collaborative scenarios due to the lack of an association mechanism, overcomes the limitations of traditional methods that rely on static spatial information of a single frame, and significantly improves the accuracy and stability of matching in dynamic and complex environments such as target movement and occlusion by utilizing the inherent temporal continuity of video streams, thus providing a reliable technical approach for achieving accurate identity association. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] Figure 1 is an application scenario diagram of the model detection result matching method, apparatus, equipment and medium provided in the embodiments of this disclosure;

[0035] Figure 2 is a flowchart of a model detection result matching method provided in an embodiment of this disclosure;

[0036] Figure 3 is a flowchart of a model detection result matching method provided in yet another embodiment of this disclosure;

[0037] Figure 4 is a schematic diagram of the structure of a model detection result matching device provided in another embodiment of this disclosure;

[0038] Figure 5 is a schematic diagram of the structure of a control device provided in another embodiment of this disclosure.

[0039] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] In the safety production management of refining and chemical enterprises, video surveillance systems often need to operate multiple models in tandem, such as safety helmet detection and human key point detection, to achieve closed-loop management from the discovery of violations to the identification of specific responsible persons. However, different models output results independently, without any identification association between them, resulting in the inability to automatically and accurately match objects (safety helmets) with people (human key points). Moreover, due to the complex on-site environment, personnel and equipment are in a state of continuous interaction and movement, relying solely on the static spatial information of a single frame image is prone to mismatches, and even brief occlusion or rapid movement of the target can render any simple association attempt ineffective.

[0042] In related technologies, a common approach is to run each model independently and then supplement it with a post-processing matching strategy based on fixed rules or simple distance metrics. However, these methods lack the ability to model and analyze the continuity and regularity of target motion. When faced with real, nonlinear dynamic scenarios, they struggle to overcome problems such as unstable matching, easy loss of data, and error propagation caused by spatial information ambiguity and temporal correlation breaks. Consequently, they fail to meet the high standards of accuracy and reliability required for monitoring results in industrial settings.

[0043] The model detection result matching method, apparatus, device, and medium provided in this application capture the temporal behavioral features by extracting motion vectors from different model detection results in multiple consecutive frames. Then, a membership function is constructed to quantify and evaluate the consistency of motion patterns, and finally, matching is achieved based on this consistency metric. Thus, the matching problem is elevated from spatial correlation in a single frame to spatiotemporal consistency verification across multiple frames. By utilizing the continuity and uniqueness of motion patterns as matching criteria, the two main obstacles of occasional spatial ambiguity and temporal correlation breaks are effectively overcome.

[0044] Figure 1 is a schematic diagram of the application scenario of the model detection result matching method, device, equipment and medium provided in this application. As shown in Figure 1, in the budget processing, the server 100 will obtain the detection results input by different types of detection models 110, process them, and output the corresponding target matching result 120, thereby completing the model detection result matching processing.

[0045] It should be noted that the scenario shown in Figure 1 includes only one or a specific number of servers, detection models, and target matching results for illustrative purposes, but this disclosure is not limited to this. In other words, the number of servers, detection models, and target matching results can be arbitrary.

[0046] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0047] Figure 2 is a schematic flowchart of the model detection result matching method provided in the embodiment of this application. The main process of the model detection result matching method will be described below with reference to Figure 2:

[0048] Step S201: Based on the image to be detected, obtain the first set of detection results output by the target detection model and the second set of detection results output by the key point detection model.

[0049] The images to be detected include multiple consecutive frames.

[0050] Specifically, the execution entity of the model detection result matching method in this embodiment is a server deployed in an enterprise data center or a processor with powerful computing capabilities. This server can access the enterprise's image monitoring sensors and acquire corresponding detection image data, while simultaneously controlling multiple image detection models to collaborate in determining and outputting the detection results. For ease of explanation, it will be referred to as the server below.

[0051] In industrial safety monitoring scenarios such as refining and chemical enterprises, the detection image data that the server needs to process is mainly a continuous video stream collected by cameras.

[0052] Specifically, the server does not perform isolated analysis on a single image. Instead, it treats the video stream as a continuous sequence of multiple frames containing a time dimension, inputs it into different models for processing, obtains the detection results of different models, and matches the detection results of these models to obtain the final output detection result.

[0053] The continuous multi-frame image sequence here constitutes a basic analysis time window, the length of which can be dynamically adjusted according to the scene. For example, to balance real-time performance and stability, the number of frames corresponding to 1 to 2 seconds is often taken (if the frame rate is 25fps, then it corresponds to 25 to 50 frames).

[0054] In terms of specific models, the server can call the object detection model deployed on it. This model is usually trained based on an open-source object detection framework (such as a single-stage detection architecture) and is specifically designed to identify specific categories, such as safety helmets, work clothes, or specific equipment.

[0055] For each frame of the input image, the model outputs a set of model detection results matching the first detection results. Each result includes a bounding box (usually represented by normalized center point coordinates, width, and height) and the confidence score of its class. These results constitute the first detection result set that changes over time.

[0056] At the same time, the server will call the key point detection model in parallel to perform detection. This model may be based on the same or different open source backbone networks and is usually dedicated to identifying structural feature points of the human body or objects.

[0057] For the same frame of image, the model outputs a set of model detection results matching the second detection result model detection result matching. Each result is usually associated with a target instance (such as a human body) and the pixel coordinates of a set of key points.

[0058] For example, in human pose estimation, a set of results might include the coordinates of 17 key points such as the nose, left and right shoulders, and left and right hips. All such results from all frames constitute the second detection result set.

[0059] The two models mentioned above can each have only one model, or each model can include two or more models that perform the same task but are based on different specific algorithms. Then, the results of each model are summarized separately (for example, the detection results corresponding to the same frame image are averaged), and then further matching processing is performed using the method provided in this solution.

[0060] When the server performs this step, it must ensure that the two models are as synchronized as possible in terms of processing time, or strictly aligned through timestamps, so as to ensure that the output results of different models correspond to the same moment in the video stream, thus ensuring effective association between multiple models in the future.

[0061] In some embodiments, the server may use asynchronous pipelines or synchronous calls to obtain results and store all results in a structured manner indexed by timestamps for subsequent retrieval and calculation.

[0062] Step S202: Extract the position change information corresponding to the first detection result set and the second detection result set between adjacent frames in the image to be detected.

[0063] The position change information includes a first set of motion vectors corresponding to the first set of detection results and a second set of motion vectors corresponding to the second set of detection results.

[0064] Specifically, after obtaining the set of detection results arranged by time, the server needs to extract dynamic change information from the static location information.

[0065] The change information here is mainly extracted from the spatial displacement of the same type of detection results between adjacent frames. After quantization, this displacement becomes the motion vector.

[0066] For the first detection result (such as the safety helmet bounding box), its position is usually represented by the coordinates of the center point of the bounding box. At this time, the server will traverse the continuous frame sequence, and for each pair of adjacent frames (e.g., frame t and frame t+1), calculate the coordinate difference of the center points of the bounding boxes with the same object detection ID. The object detection ID is maintained by the model's built-in tracking module, or determined in a short time series through simple IOU (Intersection over Union, used to measure the degree of overlap between the predicted location coordinate box and the ground truth coordinate box).

[0067] This coordinate difference (Δx, Δy) constitutes the first motion vector of the target within that time interval.

[0068] The first motion vector set is obtained by summing up the first motion vectors calculated for all targets across all adjacent frames.

[0069] For the second detection result (such as a group of human key points), the definition of its location is more complex. The server cannot simply track individual key points because key points are prone to being lost or falsely detected.

[0070] In some embodiments, the server can treat all keypoints belonging to the same entity instance as a whole and calculate the displacement of the representative point of this whole between adjacent frames.

[0071] For example, the server can calculate the mean point (centroid) of the coordinates of all key points of the instance, or calculate the center point of its bounding rectangle.

[0072] Between adjacent frames, the coordinate difference of the representative point is calculated to obtain a second motion vector. By traversing all individual instances and all adjacent frame pairs, a set of second motion vectors is obtained.

[0073] Therefore, the server can abstract the local instantaneous features of the target's motion, transforming the image sequence into a vector sequence that is easier to compare patterns with. The extraction of positional change information here is essentially a differential encoding of the target's behavior in the temporal domain. By amplifying the features of the motion pattern, it can suppress absolute positional errors in single-frame detection to a certain extent, improving matching accuracy.

[0074] In practical implementation, the server can use a tracking state machine in memory and handle situations such as the appearance and disappearance of the target to ensure the continuity of motion vector calculation and realize the management of cross-frame association of the target.

[0075] Step S203: Based on the first set of motion vectors, construct a membership function to evaluate the motion consistency of the detection results.

[0076] The optimization function must include at least a cost item based on inventory holdings and stockouts.

[0077] Specifically, after obtaining the first set of motion vectors, the server needs to learn normal or typical motion patterns from them and construct a metric to evaluate how well other motions match this pattern. This evaluation of the degree of fit can be achieved through a membership function.

[0078] The membership function originates from fuzzy mathematics. Its output value is between 0 and 1, representing the degree to which the input value belongs to a certain fuzzy concept.

[0079] In this scheme, this fuzzy concept is consistent with the typical motion pattern of target A.

[0080] The server will independently construct a membership function for each first detection result target (e.g., each tracked safety helmet). The construction is based on the target's own historical motion vector data, that is, the set of magnitudes (i.e. velocity magnitudes) of all first motion vectors generated within the current analysis time window (e.g., the past N pairs of adjacent frames).

[0081] The server dynamically determines the shape and parameters of the function by analyzing the statistical properties of this set of moduli.

[0082] For example, the server can construct a triangular membership function. This is achieved by calculating the minimum value (L) of the set of moduli. min ), maximum value (L) max ) and average (L) mean Using these three values ​​as key points, a trigonometric function can be defined:

[0083] When the input modulus is less than L min or greater than L max When the input modulus is equal to L, the membership degree is 0; when the input modulus is equal to L... mean At that time, the highest membership degree is 1; in L min To L mean Between these two points, the membership degree increases linearly; in L... mean To L max Between them, the membership degree decreases linearly.

[0084] This triangular function constitutes the motion consistency evaluator for that specific target.

[0085] Due to the function parameter (L) min L mean L max The values ​​are not fixed values ​​set in advance, but are dynamically extracted from the actual behavior data of the target itself in the current time period. This makes the evaluation criteria adaptive and able to characterize different targets (such as walking people and stationary people) or the same target at different time periods (such as walking to stopping).

[0086] In practice, the server can use a sliding window to update these statistical parameters to ensure that the function reflects the latest motion state, thereby transforming the original motion data into a motion pattern template with clear mathematical expression and interpretation, so as to facilitate further quantitative evaluation.

[0087] Step S204: Input the second set of motion vectors into the membership function, and match the first detection result set with the second detection result set based on the evaluation result output by the membership function.

[0088] Specifically, the server inputs the second set of motion vectors from the key point detection model, calculated in the aforementioned steps, into the membership function corresponding to each first detection target for consistency verification.

[0089] Specifically, it is assumed that at the initial moment, the server initially associates a second detection target (human body) with each first detection target (helmet) through spatial proximity (such as calculating the Euclidean distance between the center of the helmet frame and the centroid of the human body key point).

[0090] In subsequent frames, the server inputs the magnitude of each second motion vector generated by the associated human body into the membership function corresponding to its associated safety helmet, obtaining a series of membership values. These values ​​quantitatively represent the extent to which each instantaneous movement of the human body conforms to the typical movement pattern of its associated safety helmet.

[0091] Next, the server needs to evaluate these results to see if the two associated targets match.

[0092] In some embodiments, the server can set two thresholds: a membership degree single-decision threshold α (e.g., 0.5) and an overall confidence threshold β (e.g., 70%). The server counts the proportion F of frames with membership degrees exceeding α within the analysis time window.

[0093] If the ratio F is greater than β, the server determines that the human body and the safety helmet have a highly consistent movement pattern, maintains the matching relationship between them, and assigns them the same identity ID.

[0094] Conversely, if F is less than or equal to β, the motion patterns are determined to be inconsistent, and the current matching relationship fails.

[0095] In the event of a failed match, the server will initiate a rematch mechanism, such as reverting to the principle of the closest spatial distance in the current frame, finding a new safety helmet to associate with the human body, and starting a new round of multi-frame motion consistency verification.

[0096] Ultimately, all stable matching relationships will be determined. The server can control the image output module to overlay safety helmet frames with the same ID and human keypoint skeletons with the same color or number on the video screen, thus completing the visualization.

[0097] Therefore, by using motion consistency as a temporal deep feature as the matching criterion, the instability of single-frame spatial matching is overcome, and the matching problem is transformed from whether the positions are close in a certain frame to whether the motion is synchronized over a period of time. This greatly enhances the robustness and accuracy of matching in dynamic, crowded, and occluded industrial scenarios.

[0098] The model detection result matching method provided in this application obtains the detection results of different models in multiple consecutive frames of images, extracts their positional change information between adjacent frames to obtain a set of motion vectors, constructs a membership function to evaluate motion consistency based on this, and finally completes the matching based on the evaluation result output by the function. This effectively solves the problem of difficulty in matching detection results in multi-model collaborative scenarios due to the lack of an association mechanism. It breaks through the limitations of traditional methods that rely on static spatial information of a single frame, and utilizes the inherent temporal continuity of video streams to significantly improve the accuracy and stability of matching in dynamic and complex environments such as target movement and occlusion, providing a reliable technical approach for achieving accurate identity association.

[0099] Figure 3 is a flowchart illustrating a model detection result matching method according to another embodiment of this disclosure. Based on the embodiment shown in Figure 2, the implementation process of the model detection result matching method will be described in detail below with reference to Figure 3, which specifically includes the following steps:

[0100] S301. Based on the image to be detected, obtain the first set of detection results output by the target detection model and the second set of detection results output by the key point detection model.

[0101] The images to be detected include multiple consecutive frames.

[0102] Specifically, the server first needs to process the video stream of the input image to be detected.

[0103] To balance computational overhead with temporal information density, the server does not process every frame, but instead performs frame extraction. For example, if the video frame rate is 30 fps, the server can be set to extract one frame every 5 frames (approximately 0.167 seconds) to form a sequence of images to be detected containing multiple consecutive frames, which serves as an analysis batch.

[0104] The server then inputs each frame of the image in parallel into two independently deployed models.

[0105] The first type is the object detection model (e.g., the safety helmet detection model, work clothes detection model, etc. trained based on the open-source framework YOLO architecture), which outputs the first set of detection results.

[0106] Specifically, for each frame, the first detection result set contains multiple detection boxes, each represented by normalized center coordinates (xc, yc), width w, and height h.

[0107] The server needs to convert these normalized coordinates into image pixel coordinates. The conversion formula can be expressed as:

[0108] x min =(x c -w / 2)×W,

[0109] y min =(y c -h / 2)×H,

[0110] x max =(x c +w / 2)×W,

[0111] y max =(y c +h / 2)×H,

[0112] Where W and H are the image width and height.

[0113] The second category is keypoint detection models (e.g., human pose estimation models trained on the same or different open-source frameworks), which output a second set of detection results.

[0114] Specifically, for each frame, the set contains several human body instances, and each human body instance is defined by a set of keypoint coordinates P. (i) express:

[0115] P (i) ={(x i1 y i1 ), (x i2 y i2 ), ..., (x in y in )},

[0116] Where n is the number of key points.

[0117] The server needs to use auxiliary algorithms, either internal or external to the model (such as open-source implementations based on Part Affinity Fields, i.e., PAFs, or a combination of algorithms such as Kalman filtering and Hungarian algorithm), to correctly group all detected keypoints into different individual instances, ensuring that each P(i) represents an independent human body.

[0118] Ultimately, the server obtains two timestamp-aligned result sequences for subsequent processing.

[0119] S302. Based on the initial frame in the image to be detected, determine the distance between the first position in the first detection result and the second position in the second detection result.

[0120] The first detection result set includes the first detection result corresponding to each frame of the image to be detected, and the second detection result set includes the second detection result corresponding to each frame of the image to be detected.

[0121] Specifically, the server treats multiple consecutive frames of images to be processed as an inference unit and selects the first frame of the inference unit (i.e., multiple consecutive frames) as the initial frame.

[0122] For each first detection result in this frame (such as a helmet frame), its first position is usually defined as the center point of the detection box, i.e., ((x min +x max ) / 2, (y min +y max ) / 2).

[0123] For each second detection result (such as a human keypoint group), its second position needs to be a single point coordinate that can represent the spatial distribution of the entire keypoint set, i.e., the set position.

[0124] Furthermore, the location of the set can be obtained in the following way:

[0125] Step A1: Determine the extreme points of at least two key points contained in the second detection result in the image coordinate system.

[0126] Specifically, the server will parse the second detection result P. (i) The coordinates of all key points in the middle.

[0127] In practice, to efficiently describe the spatial occupancy of the human body, the server finds the extreme values ​​of these coordinates in the x-axis and y-axis directions.

[0128] For example, the server can calculate:

[0129] x left =min(x i1 x i2 , ..., x in ), x right =max(x i1 x i2 , ..., x in ),

[0130] y top =min(y i1 y i2 , ..., y in ), y bottom =max(y i1 y i2 , ..., y in ).

[0131] These four extreme points (x) left y top ), (x right y top ), (x left y bottom ), (x right y bottom This forms a minimum rectangular area that can encompass all the key points.

[0132] Step A2: Based on the coordinates of the extreme points, calculate the set position representing the spatial range of the second detection result.

[0133] Specifically, after obtaining the extreme points, the server needs to calculate a center point to represent the entire set. The most direct and effective way is to calculate the center of the virtual rectangle, and the specific calculation formula can be expressed as:

[0134] x interaction =( x left + x right ) / 2,

[0135] y interaction =( y top + y bottom / 2.

[0136] Where, point (x) interaction y interaction This is the location of the human body.

[0137] The server then calculates the Euclidean distance between the set location and the center point (first location) of each first detection result in the current frame for the next matching step.

[0138] S303. Based on the principle of minimum distance, determine the initial matching result between the first detection result and the second detection result.

[0139] Specifically, after obtaining the distances between all first-position-second-position pairs, the server needs to establish the initial correspondence between them to obtain the initial matching results.

[0140] The server can use the minimum distance principle for greedy matching or optimal allocation.

[0141] Specifically, for each second detection result (human body) in the current initial frame, the server finds the first detection result (object, such as a safety helmet) that has the closest Euclidean distance to its set location. If this closest distance is less than a preset reasonable threshold (e.g., no more than 20% of the image height), then the pair (human body, safety helmet) is marked as an initial match result, Z. i Specifically, it can be expressed as:

[0142] .

[0143] If multiple second detection results compete for the same first detection result, the one with the smallest distance is selected. The second detection result that is not selected will attempt to match the next closest first detection result, or be temporarily marked as unmatched.

[0144] Therefore, a temporary matching ID based on single-frame spatial information can be assigned to each spatially adjacent person-object pair for verification in subsequent multi-frame tests.

[0145] S304. Treat multiple consecutive frames of images as a single inference unit, and determine each pair of adjacent frames in the inference unit in turn.

[0146] Specifically, an inference unit refers to a set of consecutive image frames used for a single match integrity assessment, with a length T that is a configurable parameter (e.g., T = 10 frames). In this case, the server treats the T frames of images to be processed as a single analysis unit.

[0147] Next, within this unit, the server needs to analyze the continuous changes between frames.

[0148] Therefore, the server sequentially determines each pair of adjacent frames in the inference unit. That is, if the frame index is t=1,2,...,T, then the adjacent frame pairs are (1,2),(2,3),...,(T-1,T).

[0149] Thus, an inference unit containing T frames will generate T-1 pairs of adjacent frames.

[0150] Each pair of adjacent frames provides an opportunity to compute motion vectors, thus enabling the characterization of the target's motion state within that short time interval.

[0151] This analysis method based on adjacent frame pairs can effectively capture the instantaneous direction and velocity of motion, and provide sufficient data samples for subsequent construction of motion pattern statistics.

[0152] S305. For each pair of adjacent frames, based on the initial matching result, extract the motion vectors of the matching first detection result and second detection result between the pair of adjacent frames.

[0153] In this context, the motion vector of the first detection result in the adjacent frame is the first motion vector, and the motion vector of the second detection result in the adjacent frame is the second motion vector.

[0154] Specifically, for each pair of adjacent frames determined in step S304 (denoted as frame t and frame t+1), the server needs to calculate the motion of each initial matching result established in S303 within that time interval.

[0155] For a specific matching pair (e.g., an object with ID k and the matching human body): First, the server needs to find the object and the human body with ID k in the two adjacent frames. This relies on the tracking algorithms within each model (such as Kalman filtering and the Hungarian algorithm implemented based on open-source code) to maintain the cross-frame ID continuity of their respective detection results.

[0156] Then, calculate the motion vector:

[0157] First motion vector (for object motion): Calculates the coordinate difference between the center point of the detection frame of the safety helmet in frame t+1 and frame t. That is:

[0158] .

[0159] Second motion vector (human motion): Calculate the coordinate difference between the (t+1)th and tth frames for the group position of the human body (calculation method is the same as step A2 in S302). That is:

[0160] .

[0161] This calculation process is repeated on each pair of adjacent frames within the inference unit until all calculations are completed.

[0162] S306. Based on the motion vectors extracted from all adjacent frames within the inference unit, a first set of motion vectors and a second set of motion vectors are obtained.

[0163] Specifically, after completing the traversal of all adjacent frame pairs within the pair inference unit, the server will summarize the data.

[0164] For each initial matching result (ID k), calculate its first motion vector V on all T-1 pairs of adjacent frames. O (k，1) V O (k，2) , ..., V O (k，T-1) These are combined to form the first set of motion vectors for that specific object.

[0165] Similarly, its corresponding second motion vector V P (k，1) V P (k，2) , ..., V P (k，T-1) These are combined to form a second set of motion vectors specific to that human body.

[0166] From a global perspective, the union of the first motion vector sets of all initial matching results constitutes the global first motion vector set of this inference unit; similarly, the global second motion vector set is obtained.

[0167] These sets are the results of transformations from the original image coordinates to a vector space that better reflects the dynamic behavior of the target.

[0168] S307. Determine the magnitude of the first motion vector of each first detection result within the inference unit.

[0169] Specifically, in order to construct a membership function that uses the magnitude of motion speed as a metric, the server needs to extract scalar information from the set of motion vectors of each first detection result (helmet).

[0170] For the first set of motion vectors {V} belonging to a specific object k obtained in the preceding steps O (k，T) The server calculates the magnitude (i.e., L2 norm) of each motion vector.

[0171] For a motion vector V O (k，T) =(Δx, Δy), whose modulus L O (k，t) satisfy:

[0172] ,

[0173] This module length L O (k，t) The physical meaning of is the instantaneous velocity (pixels / frame) of the object during the time interval from frame t to t+1.

[0174] The server calculates a modulus sequence for each object k:

[0175] .

[0176] This sequence describes the change in the target's velocity throughout the entire inference unit time window.

[0177] S308. Based on the modulus, dynamically determine the parameters of a triangular membership function.

[0178] The membership function is a trigonometric membership function.

[0179] Specifically, the server performs statistical analysis on the modulus sequence of each obtained object k, and dynamically determines three key parameters of a triangular membership function:

[0180] Parameter a (corresponding to the left vertex of the object's triangle): Takes the minimum value in the sequence of modulo lengths, i.e.:

[0181] .

[0182] This represents the minimum instantaneous velocity of the target within the observation window.

[0183] Parameter c (corresponding to the right vertex of the object's triangle): Takes the maximum value in the modulus sequence, i.e.:

[0184] .

[0185] This represents the target's maximum instantaneous velocity within the observation window.

[0186] Parameter b (corresponding to the vertices of the triangular peaks of the object): Takes the arithmetic mean of the modulo-length sequence, that is:

[0187] .

[0188] This represents the average speed of the target within the observation window.

[0189] These three parameters (a, b, c) are derived entirely from the target's own historical observation data, and are therefore dynamically determined.

[0190] They define a support set (from a to c) and a peak point (b) of a triangular fuzzy set, which can be semantically interpreted as the typical and reasonable range of motion speeds of the target.

[0191] S309. Construct triangular membership functions based on parameters.

[0192] The triangular membership function is used to characterize the degree to which a given modulus belongs to the normal motion range defined by the motion pattern of the first detection result.

[0193] Specifically, using the defined parameters (a, b, c), the server constructs an independent triangular membership function μ for each first detection result (object k). k (z). The mathematical definition of this function can be expressed as:

[0194] ,

[0195] The input z of this function is the magnitude (velocity value) of a motion vector, and the output μ is... k (z) is a value between 0 and 1, representing the degree to which the velocity value belongs to the typical motion pattern of object k. A value of 1 indicates complete membership (velocity equal to average velocity b), and a value of 0 indicates no membership (velocity less than minimum velocity a or greater than maximum velocity c). Between a and c, the membership degree varies linearly.

[0196] This triangular membership function models the target's motion pattern in a fuzzy and quantitative way.

[0197] S310. For each initial matching result, input the magnitude of its second motion vector within the inference unit into the membership function and output the corresponding membership value.

[0198] Specifically, after obtaining the triangular membership function, the server needs to verify whether the initial matching results obtained in the aforementioned steps are reliable.

[0199] For each initial matching result (assuming it's a match between object k and human body m), the server retrieves the second set of motion vectors corresponding to that human body m and calculates the magnitude of each vector to obtain the sequence of magnitudes of human body m:

[0200] .

[0201] Then, the server will take each module length L of the human body m. P (m，t) As input z, it is substituted into the triangular membership function μ of its matching object—object k. k In (z), a series of membership values ​​were calculated:

[0202] .

[0203] Each value μ k (L P (m，t) This quantifies the extent to which the motion speed of human body m resembles the motion pattern of object k during the t-th time interval.

[0204] S311. Determine the proportion of membership values ​​greater than the preset first threshold to all membership values.

[0205] Specifically, after obtaining the membership value sequence of human body m relative to object k, the server needs to make a comprehensive judgment. Here, the first threshold is introduced—the single-match confidence threshold α (for example, let α=0.6).

[0206] The server iterates through the above membership value sequence and counts the number of values ​​greater than α, denoted as: Count valid Then, calculate the effective proportion F:

[0207] .

[0208] Where T-1 is the total number of membership values ​​(i.e., the number of adjacent frame pairs within the inference unit); the proportion F indicates the proportion of time periods within the observation time window during which the motion patterns of the human body and objects are highly consistent (single confidence exceeding α).

[0209] S312. If the ratio is greater than the preset second threshold, determine that the current initial matching result is valid and maintain the matching relationship; otherwise, determine that the current initial matching result is a matching failure.

[0210] Specifically, the server then uses a second threshold—the overall match confirmation threshold β (for example, let β = 0.75)—to make a final judgment on the proportion F:

[0211] If F > β: This indicates that for the vast majority of the time period, the movement of human body m is highly consistent with the movement pattern of object k. In this case, the server determines that the initial matching result is reliable and valid. The server will maintain the matching relationship between object k and human body m and assign or confirm a common identity ID for them.

[0212] Otherwise (F≤β): This indicates insufficient consistency in the motion patterns of the two. This may stem from an incorrect initial spatial match (e.g., the helmet or item k belongs to another person behind), or occlusion causing tracking drift. The server determines this initial match result as a failure.

[0213] Furthermore, in the event of a matching failure, the server needs to perform the following steps to ensure the matching process is completed:

[0214] Step B1: Based on the current frame in the image to be detected, redetermine the distance between the second position of the second detection result that failed to match and the first position of all first detection results.

[0215] Specifically, when a matching pair (such as human body m and object k) is determined to be unmatched, human body m becomes unmatched.

[0216] In order to find a potentially correct matching object for it, the server falls back to a spatial distance-based strategy.

[0217] However, at this point, the server does not directly roll back to the beginning, but instead uses the latest information from the last frame of the current inference unit (i.e., the current frame).

[0218] The server calculates the set position of human body m in the current frame and calculates the distance between that position and the center point of the first detection result (object) in the current frame that is not occupied by other valid matches.

[0219] Step B2: Based on the minimum distance principle, reassign the matching first detection result to the second detection result that failed to match, and redetermine the corresponding evaluation result.

[0220] Specifically, based on the latest distance calculated in the previous step, the server again applies the minimum distance principle to assign a new, closest object (let's say object p) to the human body m as the new initial matching result.

[0221] This new matching pair will be included in the analysis cycle of the next inference unit (i.e., starting from the next frame, as a new initial match), and the process of steps S304 to S312 will be repeated to verify motion consistency.

[0222] This failure-rematch mechanism forms a closed loop, enhancing the system's ability to self-correct in dynamic scenarios and ensuring the long-term stability of multi-model detection result matching.

[0223] The model detection result matching method provided in this disclosure solves the problem of unstable matching caused by the fragmentation of spatiotemporal information in dynamic scenes by establishing initial spatial association, extracting multi-frame motion features, constructing adaptive membership functions, and implementing a closed-loop decision mechanism. This elevates the matching criterion from single-frame static position to multi-frame motion mode consistency verification, significantly enhancing matching accuracy and system robustness under complex conditions such as target movement, interaction, and brief occlusion, and achieving continuous and reliable alignment of cross-model identity identifiers.

[0224] Figure 4 is a schematic diagram of a model detection result matching device provided in an embodiment of this disclosure. As shown in Figure 4, the model detection result matching device 400 includes:

[0225] The determination module 410 is used to obtain a first set of detection results output by the target detection model and a second set of detection results output by the key point detection model based on the image to be detected, wherein the image to be detected includes multiple consecutive frames of images;

[0226] The extraction module 420 is used to extract position change information between adjacent frames in the image to be detected, corresponding to the first detection result set and the second detection result set. The position change information includes the first motion vector set corresponding to the first detection result set and the second motion vector set corresponding to the second detection result set.

[0227] Module 430 is used to construct a membership function for evaluating the motion consistency of the detection results based on the first set of motion vectors.

[0228] The processing module 440 is used to input the second set of motion vectors into the membership function and match the first detection result set and the second detection result set based on the evaluation result output by the membership function.

[0229] Optionally, the determining module 410 is specifically used to: if the first detection result set includes a first detection result corresponding to each frame of the image to be detected, and the second detection result set includes a second detection result corresponding to each frame of the image to be detected, then after obtaining the first detection result set and the second detection result set, based on the initial frame in the image to be detected, determine the distance between the first position in the first detection result and the second position in the second detection result, and based on the minimum distance principle, determine the initial matching result between the first detection result and the second detection result.

[0230] Optionally, the extraction module 420 is specifically used to: treat multiple consecutive frames of images as an inference unit, and sequentially determine each pair of adjacent frames in the inference unit; for each pair of adjacent frames, based on the initial matching result, extract the motion vectors of the matching first detection result and second detection result between the pair of adjacent frames, wherein the motion vector of the first detection result in the adjacent frame is the first motion vector, and the motion vector of the second detection result in the adjacent frame is the second motion vector; based on the motion vectors extracted from all adjacent frames in the inference unit, obtain the first motion vector set and the second motion vector set.

[0231] Optionally, the construction module 430 is specifically used to: if the membership function is a triangular membership function, determine the magnitude of the first motion vector of each first detection result within the inference unit; dynamically determine the parameters of a triangular membership function based on the magnitude; and construct a triangular membership function based on the parameters, wherein the membership function is used to characterize the degree to which a given magnitude belongs to the normal motion range defined by the motion pattern of the first detection result.

[0232] Optionally, the processing module 440 is specifically used to: for each initial matching result, input the magnitude of its second motion vector in the inference unit into the membership function and output the corresponding membership value; determine the proportion of membership values ​​greater than a preset first threshold to all membership values; if the proportion is greater than a preset second threshold, determine that the current initial matching result is valid and maintain the matching relationship; otherwise, determine that the current initial matching result is a matching failure.

[0233] Optionally, the processing module 440 is further configured to, after determining that the current initial matching result is a failed match, redetermine the distance between the second position of the failed second detection result and the first position of all first detection results based on the current frame in the image to be detected; and, based on the minimum distance principle, reassign the matched first detection results to the failed second detection results and redetermine the corresponding evaluation results.

[0234] Optionally, the determining module 410 is further configured to, if the second position in the second detection result is a set position, obtain the set position based on the following method: determine the extreme points of at least two key points contained in the second detection result in the image coordinate system; and calculate the set position representing the spatial range of the second detection result based on the coordinates of the extreme points.

[0235] In this embodiment, the model detection result matching device solves the problem in related technologies where the detection results of different models are difficult to match, thus affecting the effectiveness of production safety detection, by combining various modules.

[0236] Figure 5 is a schematic diagram of the structure of a control device provided in an embodiment of the present disclosure. As shown in Figure 5, the control device 500 includes a memory 510 and a processor 520.

[0237] The memory 510 stores a computer program that can be executed by at least one processor 520. This computer program is executed by at least one processor 520 to enable the control device to implement the model detection result matching method provided in any of the above embodiments.

[0238] The memory 510 and the processor 520 can be connected via a bus 530.

[0239] The relevant explanations can be understood by referring to the corresponding descriptions and effects in the method embodiments, and will not be repeated here.

[0240] One embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the model detection result matching method provided in any of the above embodiments.

[0241] The computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0242] One embodiment of this disclosure provides a computer program product comprising computer execution instructions that, when executed by a processor, are used to implement the model detection result matching method provided in any of the above embodiments.

[0243] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0244] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0245] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0246] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for matching model detection results, characterized in that, include: Based on the image to be detected, a first set of detection results output by the target detection model and a second set of detection results output by the keypoint detection model are obtained, wherein the image to be detected includes multiple consecutive frames; position change information corresponding to the first set of detection results and the second set of detection results between adjacent frames in the image to be detected is extracted, wherein the position change information includes a first set of motion vectors corresponding to the first set of detection results and a second set of motion vectors corresponding to the second set of detection results; based on the first set of motion vectors, a membership function for evaluating the motion consistency of the detection results is constructed; the second set of motion vectors is input into the membership function, and based on the evaluation result output by the membership function, the first set of detection results and the second set of detection results are matched.

2. The method according to claim 1, characterized in that, The first detection result set includes a first detection result corresponding to each frame of the image to be detected, and the second detection result set includes a second detection result corresponding to each frame of the image to be detected; After obtaining the first set of detection results output by the target detection model and the second set of detection results output by the key point detection model based on the image to be detected, the method further includes: determining the distance between the first position in the first detection result and the second position in the second detection result based on the initial frame in the image to be detected. Based on the principle of minimum distance, the initial matching result between the first detection result and the second detection result is determined.

3. The method according to claim 2, characterized in that, The step of extracting the positional change information corresponding to the first detection result set and the second detection result set between adjacent frames in the image to be detected includes: taking the continuous multi-frame image as an inference unit, and sequentially determining each pair of adjacent frames in the inference unit; for each pair of adjacent frames, based on the initial matching result, extracting the motion vectors of the matching first detection result and second detection result between the pair of adjacent frames, wherein the motion vector of the first detection result in the adjacent frame is the first motion vector, and the motion vector of the second detection result in the adjacent frame is the second motion vector; and obtaining the first motion vector set and the second motion vector set based on the motion vectors extracted from all adjacent frames in the inference unit.

4. The method according to claim 3, characterized in that, The membership function is a triangular membership function. The step of constructing a membership function for evaluating the motion consistency of detection results based on the first set of motion vectors includes: determining the magnitude of the first motion vector of each first detection result within the inference unit; dynamically determining the parameters of a triangular membership function based on the magnitude; and constructing the triangular membership function based on the parameters. The triangular membership function characterizes the degree to which a given magnitude belongs to the normal motion range defined by the motion pattern of the first detection result.

5. The method according to claim 4, characterized in that, The matching of the first detection result set and the second detection result set based on the evaluation result output by the membership function includes: for each initial matching result, inputting the magnitude of its second motion vector within the inference unit into the membership function and outputting the corresponding membership value; determining the proportion of membership values ​​greater than a preset first threshold to all membership values; if the proportion is greater than a preset second threshold, determining that the current initial matching result is valid and maintaining the matching relationship; otherwise, determining that the current initial matching result is a matching failure.

6. The method according to claim 5, characterized in that, After determining that the current initial matching result is a failed match, the method further includes: based on the current frame in the image to be detected, re-determining the distance between the second position of the failed second detection result and the first position of all first detection results; based on the minimum distance principle, reassigning the first matching result to the failed second detection result, and re-determining the corresponding evaluation result.

7. The method according to any one of claims 2 to 6, characterized in that, The second position in the second detection result is a set position, which is obtained based on the following method: determining the extreme points of at least two key points contained in the second detection result in the image coordinate system; and calculating the set position representing the spatial range of the second detection result based on the coordinates of the extreme points.

8. A model detection result matching device, characterized in that, The model detection result matching device includes: a determination module, used to obtain a first detection result set output by a target detection model and a second detection result set output by a key point detection model based on the image to be detected, wherein the image to be detected includes multiple consecutive frames; an extraction module, used to extract position change information corresponding to the first detection result set and the second detection result set between adjacent frames in the image to be detected, wherein the position change information includes a first motion vector set corresponding to the first detection result set and a second motion vector set corresponding to the second detection result set; a construction module, used to construct a membership function for evaluating the motion consistency of the detection results based on the first motion vector set; and a processing module, used to input the second motion vector set into the membership function and match the first detection result set and the second detection result set based on the evaluation result output by the membership function.

9. A control device, characterized in that, include: Memory, processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images