A metal mold tracking method based on local directional features and correlation matrix
Through the methods of local directional features and correlation matrix, the difficulty of positioning tracking caused by the diverse internal states of the metal mold is solved, high accuracy and real-time performance are achieved, and three-dimensional trajectory is provided to support the efficient operation of industrial robots.
Patent Information
- Application Number
- CN202210155145.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-21
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-02-21
AI Technical Summary
In complex industrial environments, the diverse internal states of metal molds lead to difficulty in positioning and tracking, and existing visual processing algorithms are difficult to achieve high accuracy and real-time.
Using the metal mold tracking method based on local direction features and correlation matrix, by establishing a mold template, calculating the similarity of image feature vectors, performing non-maximum suppression and X-space correction, dividing the four states of the target mold, and completing the data correlation matrix ID annotation and tracking and map trajectories to three-dimensional space.
It achieves high accuracy, reliability and real-time in the field of industrial metal mold tracking, and supports efficient operation of industrial robots.
Smart Images

Figure CN114708298B_ABST
Abstract
Description
Technical Field
[0001] The invention specifically belongs to the technical field of image recognition, and in particular relates to a metal mold tracking method based on local directional features and an association matrix. Background Art
[0002] The nonferrous metal industry is one of the basic industries of the national economy, national defense, and science and technology. The production process of nonferrous metal ingots is to place multiple metal molds at equal intervals on a production line moving in a single direction, and then inject high-temperature liquid nonferrous metal into the mold, which solidifies into ingots after cooling. During the cooling and crystallization process, the liquid surface in contact with the air will continue to be oxidized to form oxide scum. If it is not removed in time, the quality of the solid metal ingot will not meet the standard. At present, most of the slag removal processes on the production line of solid metal ingots are performed by slag removal robots, but only using slag removal robots according to a fixed slag removal process usually leads to unsatisfactory slag removal results. In recent years, machine vision technology has been widely used in industrial inspection to realize automated inspection for industrial production processes. Intelligent perception and information fusion of production process elements have become a trend.
[0003] Machine vision takes visual information as input, processes the input information, and uses the recognition result as the information guide for the next action. Machine vision is not only suitable for production lines with a large number of repetitive labor, but also for production workshops with relatively precise installation, detection and heavy physical labor. The machine vision recognition method is used to locate the metal casting process, providing the real-time position of the metal ingot for the slag removal robot, which helps to improve the adaptive ability of the slag removal robot and improve the work quality and efficiency of the slag removal robot. At present, the commonly used visual processing algorithms include template matching and local feature descriptors. Most of these are aimed at a single target with a certain shape and no change in appearance. However, due to the complex industrial environment, there are many internal states of the metal mold during the fine slag removal process, which makes it difficult to apply. Therefore, a metal mold tracking method based on local directional features and association matrix is proposed. The algorithm establishes a template through local directional features and matches the position of the target mold in each frame. Then, inter-frame association is performed to obtain the motion trajectory of each mold, and the trajectory is mapped to three-dimensional space according to the acquired depth information, providing the robot with a three-dimensional trajectory of the mold operation. Summary of the invention
[0004] The purpose of the present invention is to address the problem of difficulty in positioning and tracking when the mold contains metals in different states, propose a metal mold tracking method based on local directional features and association matrix, obtain the motion trajectory of the mold in space, and ultimately achieve higher accuracy, reliability and real-time performance in the field of industrial metal mold tracking, providing technical support for the work of industrial robots.
[0005] In order to achieve the above technical purpose, the technical solution of the present invention is:
[0006] A metal mold tracking method based on local directional features and an association matrix comprises the following steps:
[0007] S1: Collect image data of multiple single molds on the production line and extract the common features of each mold image to establish a mold template for tracking the mold;
[0008] S2: Obtain the image of the mold to be inspected from the production line, then calculate the similarity between the image feature vector of the mold image and the mold template vector, screen out the target points that meet the conditions, and then obtain the position of the target mold through non-maximum suppression and X-space correction;
[0009] S3: The detection process from the target mold entering the detection range to tracking it leaving the detection range is divided into four states. The correlation matrix between the target frames is obtained based on the data relationship between the frames. The jump of the target mold state is completed in combination with the correlation matrix, and then the ID of the target mold is marked and tracked.
[0010] S4: Output the ID, status and position of the target mold in the current frame, transform the coordinates of the target mold in the tracking state into three-dimensional space, and provide operation coordinate points for the robot.
[0011] The metal mold tracking method based on local directional features and association matrix, the step S1 comprises:
[0012] S101: Establishing a camera coordinate system with a fixed camera position C}, then use the camera to collect image data of multiple single molds on the production line, and establish the conversion relationship between the pixel coordinate system of the mold and the camera coordinate system:
[0013]
[0014] Where u0 and v0 are the centers of the image plane, f is the focal length of the camera, (X C ,Y C ,Z C ) represents the coordinates of the mold in the camera coordinate system, and (u,v) is the pixel coordinates of the mold;
[0015] S102: Select a clear mold image, use a 5×5 convolution kernel to perform Gaussian filtering to remove noise points, then extract the gradient direction features of the mold outer contour, and use the Sobel operator to extract the gradient of the image;
[0016] S103: Filter and fit the gradient directional features of the contours on the horizontal X-axis of each mold image, so as to establish a feature template of the mold:
[0017] m d (x,y)=(m dx (x,y),m dy (x,y) T
[0018] Among them, (x, y) represents the pixel coordinates, m d (x, y) represents the gradient set of the template image, m dx (x,y) represents the gradient set of the template image in the x direction, m dy (x,y) represents the gradient set of the template image in the y direction, and the superscript T represents the matrix transpose.
[0019] The metal mold tracking method based on local directional features and association matrix, in the step S101, when collecting image data of multiple single molds on the production line, it is necessary to collect images of the same mold in different internal states, wherein the internal states include empty, overflow and standard states; in the step S102, when selecting a clear mold image, it is necessary to select a clear mold image in the three states including empty, overflow and standard.
[0020] The metal mold tracking method based on local directional features and association matrix, the step S2 comprises:
[0021] S201: Obtain an image of the mold to be inspected, i.e., a search image, from the production line, and extract a set of image gradients corresponding to the mold template. Then, the inner product s(x, y) between the search image and the corresponding unit vector in the mold template is used as the similarity:
[0022]
[0023] Among them, (t x ,t y ) represents the projection of the gradient direction vector of the point on the feature line segment on the X-axis and Y-axis, and n represents the number of points on the feature line segment; r d (x,y) represents the gradient set of the search image;
[0024] The points in the search image whose calculated s(x, y) values are greater than a preset threshold are taken as candidate points;
[0025] S202: After obtaining the candidate points, non-maximum suppression is performed. For the candidate points that appear in the form of local clusters on the image, each cluster is selected with a circular frame with a preset radius, and a point (x', y') with the highest similarity s in each cluster is extracted as the mold detection result, and a detection frame is obtained according to the detection result;
[0026] S203: Correct the detection frame of the mold: delete the detection results that deviate from other detection results by more than a preset threshold on the X-axis, that is, the movement direction of the production line.
[0027] In the metal mold tracking method based on local directional features and association matrix, in step S201, extracting the image gradient set of the search image includes the following steps:
[0028] r d (x,y)=(r dx (x,y),r dy (x,y) T
[0029] r d (x, y) represents the gradient set of the search image, r dx (x,y) represents the gradient set in the x direction of the search image, r dy (x,y) represents the gradient set of the search image in the y direction, and the superscript T represents the matrix transpose.
[0030] In the metal mold tracking method based on local directional features and association matrix, in step S3, the detection process from the target mold entering the detection range to the detection process leaving the detection range is divided into four states, which is carried out in the following manner:
[0031] The four states of the target mold include activated, tracked, lost and deactivated; whenever a new mold is detected in an image, the mold is taken as the target mold and enters the "activated" state; if the target mold that enters the "activated" state continues to be detected from subsequent images, the state is converted to "tracked" and the target mold is marked with an ID; if the target mold that enters the "activated" or "tracked" state is no longer detected from subsequent images, the state is converted to "lost"; if the target mold in the "lost" state is still not detected after the preset loss time, it enters the "deactivated" state and it is considered that the target mold has left the detection range; if the target mold in the "lost" state is detected again within the preset loss time, the state of the target mold is converted to the state before entering the "lost" state.
[0032] In the metal mold tracking method based on local directional features and association matrix, in step S3, the ID labeling and tracking of the target mold includes:
[0033] In step S3, the ID labeling and tracking of the target mold includes:
[0034] S301: After obtaining the detection results of two frames of images, a data association matrix is established, where N represents the mold detected in the previous frame and the mold in the lost state, and M represents the mold detected in the current frame. The expression a is calculated ij To represent the matching value between N and M:
[0035]
[0036] Where n represents the number of molds in N, and m represents the number of molds in M; the data association matrix is an n×m-order matrix used to represent the target similarity, a ij The value range is -∞~1, β represents the speed compensation value, that is, the average speed in the first 5 frames is taken as the predicted value of the speed in the next 5 frames, and the initial value of β is 0. i represents the ordinate of the i-th mold in N, t i Indicates the frame number of the i-th mold in N; y j represents the ordinate of the jth mold in M, t j represents the frame number of the jth mold in M;
[0037] S302: For target i, in a i1 ~a im Select the maximum similarity value a imax is the matching result; after matching, the state transfer judgment is performed; the targets in the activated and tracked states will be judged for similarity, and the targets in the lost state will be judged for similarity and loss time; the state conversion is completed according to the judgment result, and the judgment condition is shown in the following formula:
[0038]
[0039] Among them, the similarity threshold is L, the time threshold is T; similarity matching success is LT, matching failure is LF, not exceeding the maximum loss time is TT, and exceeding the maximum loss time is TF.
[0040] In the metal mold tracking method based on local directional features and association matrix, in step S4, the conversion formula for converting the coordinates of the target mold in the tracking state into the three-dimensional space is:
[0041]
[0042] Among them, (X C ,Y C ,Z C ) represents the position of the metal mold in the camera coordinate system, u0 and v0 are the centers of the image plane, (f x ,f y ) represents the focal length of the camera in the x and y directions, and (x, y) is the pixel coordinate detected by the mold.
[0043] An electronic device, comprising:
[0044] one or more processors;
[0045] a storage device for storing one or more programs,
[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0047] A computer readable medium stores a computer program, which implements the above method when executed by a processor.
[0048] The technical effect of the present invention is that a method combining local directional feature matching is proposed to locate the mold, which effectively solves the problem of difficult positioning when the metal mold contains metals in different states. Secondly, four state subspaces are set up for the metal mold, and the data association matrix is used to complete the data association in combination with the current state space of the mold, and the mold ID is marked and tracked. The tracking method combining the two can detect multiple molds at the same time and extract the running trajectories of multiple molds. It has certain advantages in accuracy, reliability and real-time performance. Finally, the obtained mold trajectory is mapped to a three-dimensional space and provided to the industrial slag removal robot to complete the slag removal action. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a schematic diagram of the process of the metal mold visual inspection and tracking method described in an embodiment of the present invention.
[0050] Figure 2 This is an image collected on a zinc ingot production line as described in an embodiment of the present invention.
[0051] Figure 3 a) is a model diagram of the mold contour designed in an embodiment of the present invention; b) is a local feature extraction diagram of the zinc ingot mold.
[0052] Figure 4 a) is the target state transition diagram designed in the embodiment of the present invention; b) is the state transition condition diagram designed in the present invention.
[0053] Figure 5 This is a diagram showing the metal mold tracking effect on a zinc ingot production line according to an embodiment of the present invention. DETAILED DESCRIPTION
[0054] The present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.
[0055] See also Figure 1 In this embodiment, Figure 2Taking the several frames of zinc ingot metal mold images collected as an example, the metal mold tracking method based on local directional features and association matrix provided in this embodiment includes the following steps:
[0056] S1: Figure 2 As shown, the inside of the metal mold contains multiple states, namely empty, overflow and standard. Empty means there is no metal solution in the mold, overflow means that the metal solution in the metal mold is higher than the upper limit of the metal mold, and standard means that the capacity of the metal solution in the metal mold is within the preset normal range. Collect multiple single mold image data on the production line, and collect images of the same mold in different internal states, then denoise the image, extract the image gradient, construct the gradient image, extract the local same direction features of these molds, analyze, filter and fit the same direction features, and configure the mold template with the local feature vector.
[0057] Specifically, step S1 includes:
[0058] S101: First install the industrial camera and establish a Cartesian space coordinate system, i.e., the camera coordinate system {O C}, the camera keeps a fixed position, fully collects the image data of the metal mold on the production line, and establishes the image coordinate system.
[0059] S102: After selecting clear mold images belonging to different states, firstly use Gaussian filtering to remove noise points, then extract the gradient direction features of the mold outer contour, and use the Sobel operator to extract the gradient of the image.
[0060] S103: By Figure 3 As shown in a), the eight external contours of the mold are analyzed one by one. Through detection, it is found that the AB segment and FE segment of different molds have obvious similarities in the gradient direction. This is because there is a gap between the molds, so the lateral contours generally have similar characteristics in the gradient direction. However, the gradient direction characteristics of the other 6 edges are relatively vague. The points on the AB segment have the feature of the gradient direction vertically upward, and the points on the FE segment have the feature of the gradient direction vertically downward, as shown in Figure 1. Figure 3 b). Filtering and fitting are performed in this way, and the feature of the upper edge of the metal mold is selected as a unit vector with a vertical upward direction, and the feature of the lower edge is selected as a unit vector with a vertical downward direction. The template feature image of the metal mold is established by the correspondence between the AB segment, the FE segment and the other 6 line segments. The original image to be matched is also Gaussian filtered using the same convolution kernel, and the image gradient is extracted using the Sobel operator, and the gradient set of the image is recorded.
[0061]
[0062] Among them, m d (x, y) represents the gradient set of the template image, rd (x,y) represents the gradient set of the search image. (x,y) represents the pixel coordinates, m d (x, y) represents the gradient set of the template image, r d (x, y) represents the gradient set of the search image, m dx (x,y) represents the gradient set of the template image in the x direction, m dy (x,y) represents the gradient set of the template image in the y direction, and the superscript T represents the matrix transpose.
[0063] S2: Based on the high consistency of mold feature distribution, the similarity between the current image feature vector and the template vector is calculated using relevant parameters in statistical techniques, and the target points that meet the conditions are screened out. Then, the position of the target mold is determined through non-maximum suppression and X-space correction.
[0064] S201: The similarity measure is defined based on the inner product between the corresponding unit vectors in the template image and the search image, and is defined as s(x, y). It represents the similarity between the point at position (x, y) in the search image and the template image. After normalization, s takes the value of 0 to 1. A threshold needs to be set for the matching result of the local gradient direction feature. When the threshold is set too high, it may not be detected; when it is set too low, there will be too many interference items affecting the detection of the mold. After many experiments, it was determined that the candidate point performed best when s>0.9, so the matching threshold of 0.9 was selected in this embodiment.
[0065]
[0066] Among them, (t x ,t y ) represents the projection of the gradient direction vector of the point on the feature line segment on the X-axis and Y-axis, and n represents the number of points on the feature line segment.
[0067] S202: After obtaining the candidate points, non-maximum suppression is performed. Non-maximum suppression is to select the point with the highest similarity within a certain local range as the mold detection result. The local range in this embodiment is selected as 10 pixel coordinates, that is, each cluster is selected with a circular frame with a radius of 10 pixels, and a point (x′, y′) with the highest similarity s in each cluster is extracted as the mold detection result.
[0068]
[0069] S203: X-axis correction is based on the fact that the movement direction of the production line is perpendicular to the movement direction of the Y-axis, so theoretically the X-axis coordinates of all molds have very high similarity. Based on this factor, the mold detection frame is corrected, and the X-coordinates that are obviously deviated are set as abnormal detections and deleted. In this embodiment, the deviation of more than 20 pixels from other mold detection results is used as the threshold for abnormal detection.
[0070] S3: The mold detection process is divided into four state subspaces. The correlation matrix between target frames is obtained from the inter-frame data relationship. The mold ID state jump is completed in combination with the correlation matrix, and then labeling and tracking are performed to effectively deal with occlusion and false detection.
[0071] S301: Figure 4 As shown in a), the four states of the target mold include activated, tracked, lost and deactivated. Whenever a new mold is detected in the image, the mold is taken as the target mold and enters the "activated" state. If the target mold that enters the "activated" state continues to be detected from subsequent images, the state is converted to "tracked" and the target mold is marked with an ID. If the target mold that enters the "activated" or "tracked" state is no longer detected from subsequent images, for example, it disappears from the camera's field of view due to being blocked or other reasons, the state is converted to "lost". If the target mold in the "lost" state is still not detected after the preset loss time, it enters the "deactivated" state. If the target mold in the "lost" state is re-detected within the preset loss time, the state of the target mold is converted to the state before entering the "lost" state. Among them, "deactivated" is the terminal state of any target. The target marked with an ID entering this state indicates that the target has left the shooting range.
[0072] S302: After obtaining the detection results of the two frames of images, a data association matrix is established. The data association matrix is an n×m-order matrix used to represent the target similarity, where n and m represent the number of target detections in the previous and next frames, respectively. Here, N represents the mold detected in the previous frame and the mold in the lost state, and M represents the mold detected in the current frame. The expression a is calculated ij To represent the matching value between N and M:
[0073]
[0074] Where n represents the number of molds in N, and m represents the number of molds in M; the data association matrix is an n×m-order matrix used to represent the target similarity, a ij The value range is -∞~1, β represents the speed compensation value, that is, the average speed in the first 5 frames is taken as the predicted value of the speed in the next 5 frames, and the initial value of β is 0. i represents the ordinate of the i-th mold in N, t iIndicates the frame number of the i-th mold in N; y j represents the ordinate of the jth mold in M, t j represents the frame number of the jth mold in M;
[0075] S303: For target x i , in a i1 ~a im Select the maximum similarity value a imax is the matching result. After matching, the state transition is judged by the attached Figure 4 b). Similarity judgment will be performed in the activated and tracked states, and similarity judgment and loss time judgment will be performed in the lost state. The state conversion is completed according to the judgment result, and the judgment condition is shown in the following formula:
[0076]
[0077] Among them, the similarity threshold is L, and the time threshold is T. Successful similarity matching is LT, failed matching is LF, within the maximum loss time is TT, and beyond the maximum loss time is TF.
[0078] S4: Output the ID of the current frame mold and its status and position, as shown in the attached Figure 5 As shown. The mold coordinates in the tracking state are converted into three-dimensional space to provide the robot with operation coordinate points. The conversion formula is:
[0079]
[0080] Among them, (X, Y, Z) represents the position of the metal mold in the camera coordinate system. (u0, v0) represents the pixel coordinate system, (x, y) represents the image coordinate system, and (f x ,f y ) represents the focal length of the camera in the x and y directions.
[0081] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.
[0082] Electronic equipment includes:
[0083] one or more processors;
[0084] a storage device for storing one or more programs,
[0085] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.
[0086] In specific use, users can interact with servers, which are also electronic devices, through electronic devices as terminal devices and based on the network to achieve functions such as receiving or sending messages. Terminal devices are generally various electronic devices equipped with display devices and used based on human-computer interfaces, including but not limited to smart phones, tablet computers, laptops and desktop computers. Various specific application software can be installed on the terminal devices as needed, including but not limited to web browser software, instant messaging software, social platform software, shopping software, etc.
[0087] The server is a network service end for providing various services. The method provided in this embodiment is generally executed by the server. In actual use, the terminal device can also directly execute it if necessary conditions are met.
[0088] Similarly, the computer-readable medium of the present invention stores a computer program thereon, and when the program is executed by a processor, a sales forecasting method according to an embodiment of the present invention is implemented.
[0089] So far, the technical solution of the present invention has been described in conjunction with the accompanying drawings. In this example, first, Gaussian filtering is performed on multiple single zinc ingot mold images and the local directional features of the mold are extracted. Secondly, a zinc ingot mold template is established based on the local directional features, and the target points that meet the conditions are determined in the actual collected images through similarity measurement, and then the position of the zinc ingot mold is determined through non-maximum suppression and X-axis correction. The correlation matrix between target frames is obtained through the inter-frame data relationship, and the mold ID is labeled and tracked in combination with the correlation matrix. Finally, the obtained zinc ingot mold trajectory is mapped to three-dimensional space. The algorithm proposed in the present invention has achieved high accuracy, reliability and real-time performance in the field of industrial mold recognition and detection, reflecting the superiority of the present invention in the industrial field.
Claims
1. A metal mold tracking method based on local directional features and association matrix, characterized in that: The following steps are involved: S1: Collect image data of multiple single molds on the production line and extract the common features of each mold image to establish a mold template for tracking the mold; S2: Obtain the image of the mold to be inspected from the production line, then calculate the similarity between the image feature vector of the mold image and the mold template vector, screen out the target points that meet the conditions, and then obtain the position of the target mold through non-maximum suppression and X-space correction; S3: The detection process from the target mold entering the detection range to tracking it leaving the detection range is divided into four states. The correlation matrix between the target frames is obtained based on the data relationship between the frames. The jump of the target mold state is completed in combination with the correlation matrix, and then the ID of the target mold is marked and tracked. S4: Output the ID, status and position of the target mold in the current frame, transform the coordinates of the target mold in the tracking state into three-dimensional space, and provide operation coordinate points for the robot.
2. According to claim 1, a metal mold tracking method based on local directional features and correlation matrix is characterized in that: The step S1 comprises: S101: Establishing a camera coordinate system with a fixed camera position C }, then use the camera to collect image data of multiple single molds on the production line, and establish the conversion relationship between the pixel coordinate system of the mold and the camera coordinate system: Where u0 and v0 are the centers of the image plane, f is the focal length of the camera, (X C ,Y C ,Z C ) represents the coordinates of the mold in the camera coordinate system, and (u,v) is the pixel coordinates of the mold; S102: Select a clear mold image, use a 5×5 convolution kernel to perform Gaussian filtering to remove noise points, then extract the gradient direction features of the mold outer contour, and use the Sobel operator to extract the gradient of the image; S103: Filter and fit the gradient directional features of the contours on the horizontal X-axis of each mold image, so as to establish a feature template of the mold: m d (x,y)=(m dx (x,y),m dy (x,y)) T Among them, (x, y) represents the pixel coordinates, m d (x, y) represents the gradient set of the template image, m dx (x,y) represents the gradient set of the template image in the x direction, m dy (x,y) represents the gradient set of the template image in the y direction, and the superscript T represents the matrix transpose.
3. A metal mold tracking method based on local directional features and correlation matrix according to claim 2, characterized in that: In the step S101, when collecting image data of multiple single molds on the production line, it is necessary to collect images of the same mold in different internal states, where the internal states include empty, overflow and standard. In the step S102, when selecting a clear mold image, it is necessary to select a clear mold image in the three states including empty, overflow and standard.
4. The metal mold tracking method based on local directional features and correlation matrix according to claim 2 is characterized in that: The step S2 comprises: S201: Obtain an image of the mold to be inspected, i.e., a search image, from the production line, and extract a set of image gradients corresponding to the mold template. Then, the inner product s(x, y) between the search image and the corresponding unit vector in the mold template is used as the similarity: Among them, (t x ,t y ) represents the projection of the gradient direction vector of the point on the feature line segment on the X-axis and Y-axis, and n represents the number of points on the feature line segment; r d (x,y) represents the gradient set of the search image; The points in the search image whose calculated s(x, y) values are greater than a preset threshold are taken as candidate points; S202: After obtaining the candidate points, non-maximum suppression is performed. For the candidate points that appear in the form of local clusters on the image, each cluster is selected with a circular frame with a preset radius, and a point (x', y') with the highest similarity s in each cluster is extracted as the mold detection result, and a detection frame is obtained according to the detection result; S203: Correct the detection frame of the mold: delete the detection results that deviate from other detection results by more than a preset threshold on the X-axis, that is, the movement direction of the production line.
5. A metal mold tracking method based on local directional features and correlation matrix according to claim 4, characterized in that: In step S201, extracting the image gradient set of the search image includes the following steps: r d (x,y)=(r dx (x,y),r dy (x,y)) T r d (x, y) represents the gradient set of the search image, r dx (x,y) represents the gradient set in the x direction of the search image, r dy (x,y) represents the gradient set of the search image in the y direction, and the superscript T represents the matrix transpose.
6. The metal mold tracking method based on local directional features and correlation matrix according to claim 1 is characterized in that: In step S3, the detection process from the target mold entering the detection range to tracking until it leaves the detection range is divided into four states, which is performed in the following manner: The four states of the target mold include activated, tracked, lost and deactivated; whenever a new mold is detected in the image, the mold is taken as the target mold and enters the "activated" state; If the target mold that enters the "Active" state continues to be detected from subsequent images, the state is converted to "Tracking" and the target mold is marked with an ID; if the target mold that enters the "Active" or "Tracking" state is no longer detected from subsequent images, the state is converted to "Lost"; If the target mold in the "lost" state is still not detected after the preset lost time, it enters the "deactivated" state and is considered to have left the detection range; If the target mold in the "lost" state is detected again within the preset lost time, the state of the target mold is converted to the state before entering the "lost" state.
7. A metal mold tracking method based on local directional features and correlation matrix according to claim 6, characterized in that: In step S3, the ID labeling and tracking of the target mold includes: S301: After obtaining the detection results of two frames of images, a data association matrix is established, where N represents the mold detected in the previous frame and the mold in the lost state, and M represents the mold detected in the current frame. The expression a is calculated ij To represent the matching value between N and M: Where n represents the number of molds in N, and m represents the number of molds in M; the data association matrix is an n×m-order matrix used to represent the target similarity, a ij The value range is -∞~1, β represents the speed compensation value, that is, the average speed in the first 5 frames is taken as the predicted value of the speed in the next 5 frames, and the initial value of β is 0. i represents the ordinate of the i-th mold in N, t i Indicates the frame number of the i-th mold in N; y j represents the ordinate of the jth mold in M, t j represents the frame number of the jth mold in M; S302: For target i, in a i1 ~a im Select the maximum similarity value a imax is the matching result; after matching, the state transfer judgment is performed; the targets in the activated and tracked states will be judged for similarity, and the targets in the lost state will be judged for similarity and loss time; the state conversion is completed according to the judgment result, and the judgment condition is shown in the following formula: Among them, the similarity threshold is L, the time threshold is T; similarity matching success is LT, matching failure is LF, not exceeding the maximum loss time is TT, and exceeding the maximum loss time is TF.
8. The metal mold tracking method based on local directional features and association matrix according to claim 1 is characterized in that: In the step S4, the conversion formula for converting the target mold coordinates in the tracking state into the three-dimensional space is: Among them, (X C ,Y C ,Z C ) represents the position of the metal mold in the camera coordinate system, u0 and v0 are the centers of the image plane, (f x ,f y ) represents the focal length of the camera in the x and y directions, and (x, y) is the pixel coordinate detected by the mold.
9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.
10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
