A target positioning method and system for visual tracking

Abstract

The application discloses a target positioning method and system for visual tracking, and relates to the technical field of machine vision positioning. The method comprises the following steps: splitting a target process flow to obtain a target process step set; arranging visual detection equipment based on the target process step set to form a visual detection equipment set; positioning target batch products to obtain real-time process steps; calling and activating corresponding visual detection equipment to perform image acquisition and obtain real-time production video information; positioning defective products after frame extraction of the real-time production video information to obtain defective product positioning information; and performing visual tracking on the defective products based on the real-time production video information and the defective product positioning information to obtain target defect positioning results. The method solves the technical problem that, in the prior art, some workpieces have defects in the middle steps, and still need to be removed after traversing the remaining process flow, resulting in high production cost and resource waste. The method achieves the technical effects of reducing production cost and resource waste.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision positioning, and in particular to a target positioning method and system for visual tracking. BACKGROUND

[0002] Modern society cannot do without industrialization and manufacturing, and manufacturing is closely related to assembly line production. In actual assembly line production, defective products inevitably occur. Improving the efficiency and accuracy of defect detection in assembly line production, removing defective products, and thus improving the yield of products are of great significance to improving production efficiency and ensuring quality safety, and higher requirements are put forward for defect detection in assembly line production. The existing defect detection technology based on machine vision positioning often identifies, positions and removes defective products at the end of a series of process flows. However, some workpieces may have defects in the middle steps, and still need to be removed after traversing the remaining process flows, resulting in high production cost and resource waste. SUMMARY

[0003] The purpose of the present application is to provide a target positioning method and system for visual tracking. To solve the technical problem that some workpieces have defects in the middle steps and still need to be removed after traversing the remaining process flows, resulting in high production cost and resource waste.

[0004] In view of the above technical problems, the present application provides a target positioning method and system for visual tracking

[0005] In a first aspect, the present application provides a target positioning method for visual tracking, wherein the method comprises: performing process step splitting on a target process flow to obtain a target process step set, wherein the target process step set comprises K target process steps; performing visual detection device layout based on the target process step set to form a visual detection device set, wherein the target process step set and the visual detection device set have a one-to-one mapping relationship; performing target batch product positioning based on the target process flow to obtain a real-time process step, wherein the real-time process step belongs to the target process step set; calling and activating a real-time visual detection device in the visual detection device set according to the real-time process step; performing image acquisition on the target batch product based on the real-time visual detection device to obtain real-time production video information; frame extraction is performed on the real-time production video information, and defect product positioning is performed based on real-time production frame images to obtain defect product positioning information; performing visual tracking of the defect product based on the real-time production video information and the defect product positioning information to obtain a target defect positioning result.

[0006] In a second aspect, the present application further provides a target positioning system for visual tracking, wherein the system comprises:

[0007] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0008] The present application obtains a target process step set by splitting the target process flow; arranges visual detection equipment based on the target process step set to form a visual detection equipment set, which is one-to-one corresponding to the target process step set; positions a target batch product based on the target process flow to obtain a real-time process step; activates a corresponding real-time visual detection equipment according to the real-time process step; acquires images of the target batch product based on the real-time visual detection equipment to obtain real-time production video information; frames the real-time production video information, and positions a defective product based on the real-time production frame image to obtain defective product positioning information; performs visual tracking of the defective product based on the real-time production video information and the defective product positioning information to obtain a target defective positioning result. By arranging visual detection equipment based on multiple target process steps, images of the product are acquired and positioned, thereby solving the technical problem of the prior art that some workpieces have defects in the middle step and still need to be removed after traversing the remaining process flow, resulting in high production cost and resource waste. The technical effect of reducing production cost and resource waste is achieved.

[0009] The above description is only a summary of the technical solutions of the present application. In order to more clearly illustrate the technical means of the present application, and then the content of the description can be implemented, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0010] The embodiments of the present application and the following simple description are illustrated by the drawings, and the drawings are as follows:

[0011] Figure 1 A flowchart of a target positioning method of visual tracking according to the present application;

[0012] Figure 2 A flowchart of positioning a defective product based on real-time production frame images in a target positioning method of visual tracking according to the present application;

[0013] Figure 3 A structural diagram of a target positioning system of visual tracking according to the present application.

[0014] Explanation of reference signs: process step acquisition module 11, detection equipment arrangement module 12, product positioning module 13, detection activation module 13, image acquisition module 15, defective product positioning module 16, visual tracking module 17. DETAILED DESCRIPTION

[0015] The application solves the technical problems of high production cost and resource waste caused by the fact that some workpieces have defects in the middle step and still need to be removed after traversing the remaining process flow by providing a target positioning method and system for visual tracking.

[0016] The overall idea adopted by the technical embodiment in the present application to solve the above problems is as follows:

[0017] First, the target process flow is subjected to process step splitting to obtain a target process step set. Then, visual inspection equipment is arranged based on the target process step set to form a visual inspection equipment set, and the target process step set and the visual inspection equipment set correspond one-to-one. Then, target batch products are positioned based on the target process flow to obtain real-time process steps. Then, the corresponding real-time visual inspection equipment is called and activated according to the real-time process steps. Then, the target batch products are subjected to image acquisition based on the real-time visual inspection equipment to obtain real-time production video information. Then, the real-time production video information is subjected to frame extraction and defect product positioning to obtain defect product positioning information. Finally, defect product visual tracking is performed based on the real-time production video information and the defect product positioning information to obtain a target defect positioning result. By arranging visual inspection equipment based on multiple target process steps, image acquisition of products is performed, and positioning is recognized, thereby solving the technical problems of high production cost and resource waste caused by the fact that some workpieces have defects in the middle step and still need to be removed after traversing the remaining process flow. The technical effects of reducing production cost and reducing resource waste are achieved.

[0018] To better understand the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings and specific embodiments of the specification. It should be noted that the described embodiments are only some of the embodiments of the present application, not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In addition, it should be noted that only parts related to the present application are shown in the drawings for convenience of description.

[0019] Embodiment one

[0020] As shown in Figure 1 The present application provides a target positioning method for visual tracking, wherein the method comprises:

[0021] S100: subjecting a target process flow to process step splitting to obtain a target process step set, wherein the target process step set comprises K target process steps;

[0022] Specifically, in the production line, the production and processing of products are often divided into multiple steps, and multiple steps are connected to form a complete process. For the splitting of the target process, preferably, first, traverse the target process to comprehensively analyze and understand the target process. Understand the overall structure of the process, the dependency between steps, and the function and purpose of each step. Then, according to the analysis results, determine the splitting standard of the target process. This can be divided according to function, work content, operation object, etc. Then, according to the characteristics and splitting standard of the target process, the target process is split into multiple process steps according to the splitting standard. Each process step should be an operation unit with independent function or clear purpose, and has a significant effect on the form, properties, etc. of the target product, and has the possibility of defects.

[0023] Specifically, it should be noted that the target process step set is a subset of the actual process step set in production. Exemplarily, the actual process step set in production includes M actual process steps, then K≤M. Exemplarily, taking an industrial anodic oxidation production line as an example, the process includes pretreatment, anodic oxidation, coloring, post-treatment, etc. After the step splitting, the actual process step set includes: hanging, degreasing / oil removal, washing, etching, washing, neutralization, washing, polishing, washing, neutralization, anodic oxidation, activation, dyeing, sealing, ash removal, washing, drying, hanging, etc. Among them, the actual process steps such as hanging, washing, neutralization, drying, and hanging do not have the possibility of causing defects in the product, so they do not belong to the target process step. The steps belonging to the target process step set include: degreasing / oil removal, etching, polishing, anodic oxidation, dyeing, sealing, ash removal, drying, etc. By splitting the target process, the target process step set is obtained, which achieves the technical effect of ensuring the completeness of the detection node while reducing the number of detection nodes and saving the layout cost.

[0024] S200: Based on the target process step set, a visual inspection equipment layout is performed to form a visual inspection equipment set, wherein the target process step set and the visual inspection equipment set have a one-to-one mapping relationship;

[0025] Specifically, the visual detection device refers to a device that collects images of the target product and transmits the image collection results to the visual detection through an interface. The visual detection device is composed of an image sensor, a lens, an image processing chip, a data transmission interface, etc. The image sensor types include CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor), etc. The image processing chip is used to preprocess the collected raw target product images to improve image quality and reduce noise. The preprocessing includes image denoising, image enhancement, color correction, etc. to ensure that the collected image information is clear and visible. For example, time domain filtering, spatial domain filtering or frequency domain filtering methods are used for image denoising; histogram equalization, adaptive contrast enhancement, etc. are used for image enhancement; optical flow method or block matching algorithm motion compensation.

[0026] Specifically, according to the characteristics of the target product, the detection image quality requirements, etc., the visual detection device is arranged at a suitable position after each step of the target process step set, and then for each visual detection device, the detection task and the corresponding parameter setting required according to the corresponding target process step are determined. This can include image acquisition parameters, threshold setting, etc. Through the above arrangement, the target process step set and the visual detection device set are one-to-one mapped, ensuring that each process step has a corresponding visual detection device for detection, achieving the technical effect of real-time image acquisition and detection in each process step.

[0027] S300: Based on the target process flow, the target batch product is positioned to obtain a real-time process step, wherein the real-time process step belongs to the target process step set;

[0028] Specifically, the target product positioning refers to determining the current process step of the target product according to the signals, device states or product positions on the monitoring production line, etc. Preferably, a sensor is arranged at the outlet of the process step to detect the target batch product. When the sensor detects that the target batch product passes through the outlet, a positioning signal is obtained, and the real-time process step is determined according to the positioning signal. This provides the technical effect of providing a collection target for subsequent calling of corresponding visual detection devices and collecting image data.

[0029] S400: According to the real-time process step, the active real-time visual detection device is called and activated from the visual detection device set;

[0030] Specifically: first, a plurality of visual inspection devices are marked, wherein the mark corresponds to the target process step one by one, and then according to the real-time process step, the visual inspection device corresponding to the mark in the visual inspection device set is queried and activated. Wherein, the activation includes the start, wake-up, parameter configuration, etc. of the device. By determining the visual inspection device that needs to be activated by the real-time process step, it ensures that the corresponding device is called at the appropriate time to perform image acquisition and detection. Achieve the technical effect of ensuring the consistency of the detection task with the real-time process step, improve the accuracy and efficiency of detection.

[0031] S500: based on the real-time visual inspection device, image acquisition is performed on the target batch product to obtain real-time production video information;

[0032] Specifically: the image acquisition determines the acquisition parameters according to the actual environment, including resolution, code rate, frame rate, exposure, etc. For example, for an industrial anodic oxidation assembly line, the acquisition parameters are: resolution 4096*2160, frame rate 30FPS, code rate 60Mbps.

[0033] S600: frame extraction is performed on the real-time production video information, and defect product positioning is performed based on the real-time production frame image to obtain defect product positioning information;

[0034] Specifically: the real-time production video information is composed of continuous single-frame images, in order to ensure the recognition and tracking effect, the resolution, frame rate and code rate are all high. There is a lot of repeated information or similar information between the continuous single-frame images, and the amount of invalid data is large, which causes the production video information to be large in size, not conducive to transmission, storage and subsequent defect product positioning, identification, tracking and other steps, and brings huge pressure and waste to transmission bandwidth, storage space and computing power.

[0035] Specifically, the frame extraction refers to the process of setting the coding unit size, compressing and extracting key information from the real-time production video information through video compression and encoding technology. Common video compression and encoding technologies include H.264 / AVC Advanced Video Coding (AVC), H.265 / HEVC High Efficiency Video Coding (HEVC), VP9, AV1, etc. The above video compression and encoding technologies use compression algorithms and encoding strategies including spatial domain compression, temporal domain compression, motion estimation, transform coding and entropy coding. Reduces the storage and transmission of redundant data, while maintaining acceptable video quality.

[0036] It should be understood that selecting an appropriate video compression encoding technique depends on application requirements, bandwidth limitations, and device compatibility. Different techniques may differ in compression efficiency, processing complexity, and decoding requirements. Therefore, in practical applications, it is necessary to select and optimize according to the specific situation to obtain the best video compression effect.

[0037] Further, as shown in Figure 2 The real-time production video information is frame extracted, and the defective product positioning is performed based on the real-time production frame extracted image to obtain the defective product positioning information. Step S600 includes:

[0038] S610: The real-time production video information is compressed to obtain production video compression information;

[0039] S620: Based on the production video compression information, the first I frame is extracted from the first encoding unit to obtain the first I frame;

[0040] S630: The first I frame is input into the pre-constructed defective product positioning model for defective product positioning to obtain the defective product positioning information.

[0041] Specifically, the compression processing refers to a processing process that reduces the volume of the real-time production video information through video compression encoding technology, reduces redundant data, and ensures recognizability. Preferably, the application adopts the video compression encoding technology based on H265 / HEVC to perform the compression processing to obtain the production video compression information. The main compression algorithm and encoding strategy of H.265 include: Intra Prediction, Motion Estimation and Compensation, Transform Coding, Inter Prediction, Entropy Coding, etc.

[0042] Specifically, the production video compression information includes I frames (Intra Frames), P frames (Predicted Frames), and B frames (Bidirectional Frames). Among them, the I frame is a key frame in video encoding. Each I frame is a complete and independent image frame, which is encoded without relying on information of other frames, and maintains the integrity of image quality. The P frame is a predicted frame obtained by motion estimation and compensation of a forward reference frame. The P frame only stores the difference or residual information between the reference frame, thereby realizing the compression of the image. The P frame relies on the previous I frame or P frame for decoding, and performs motion compensation with the reference frame to restore the image. The B frame is a bidirectional predicted frame obtained by motion estimation and compensation of forward and backward reference frames. The B frame stores the difference or residual information between the previous and next frames, and can achieve a higher degree of compression. The B frame needs to refer to the previous and next frames for motion compensation during decoding.

[0043] Specifically, the encoding unit refers to a collection of a unique I frame and a plurality of image frames including a plurality of P frames and B frames. The first I frame is obtained by performing I frame extraction on the first encoding unit, which is an image frame containing the initial position of the defective product. Then, the first I frame is input into the pre-constructed defective product positioning model as input. The model can be an algorithm based on machine learning or deep learning, which is used to identify and locate the defective product. Through the processing of the defective product positioning model, the defective product positioning information can be obtained, achieving the technical effect of indicating the position, type or other related information of the defective product in the image.

[0044] Further, the real-time production video information is frame extracted, and the defective product positioning is performed based on the real-time production frame extracted image to obtain the defective product positioning information. Before step S600, the method further includes:

[0045] S601: calling a first target process step based on the target process step set;

[0046] S602: obtaining a first defective image set of the first target process step by interacting with the production log;

[0047] S603: presetting a defect feature recognition rule, performing defect feature recognition on the first defective image set based on the defect feature recognition rule to obtain a first defect feature set;

[0048] S604: obtaining a target defect feature set of the target process step set in the same way.

[0049] Specifically, the production log contains all execution records, abnormal situation records, and corresponding defective image information in the target process step set. The first defective image set is the defective image information related to the first target process step.

[0050] Specifically, the preset defect feature recognition rule is determined according to the target product and the target production step set. For example, in an industrial anodic oxidation production line, first, the size, number, and area occupied by the stains and pits are identified according to the uniformity, bright spots, and color spots of the image. Then, the defect determination rule is determined according to the quality requirement of the target product, including the number and length. Then, the defects that meet the defect determination rule are set and stored to obtain the first defect feature set. By traversing the target process step set, the target defect feature set is obtained, and the technical effect of providing identification parameters for subsequent defect visual tracking is achieved.

[0051] Further, the first I frame is input into the pre-constructed defect product positioning model for defect product positioning, and step S630 further includes:

[0052] S630-1: The defect product positioning model includes an input layer, an image segmentation layer, a defect recognition layer, and an output layer.

[0053] S630-2: The defect recognition layer includes K defect positioning sub-models, wherein the K defect positioning sub-models are one-to-one mapped with the K target process steps.

[0054] S630-3: Interact with the production log to obtain historical transfer image data of the same model product as the target batch product, and generate a sample transfer image set.

[0055] S630-4: Segment and identify the conveyor belt background image and the product image in the sample transfer image set to obtain a sample transfer image segmentation result.

[0056] S630-5: Construct the image segmentation layer based on the sample transfer image set and the sample transfer image segmentation result.

[0057] S630-6: According to the K target process steps, the sample transfer image segmentation result is split into K groups of sample transfer image segmentation results.

[0058] S630-7: According to the target defect feature set, the K groups of sample transfer image segmentation results are identified for defects to obtain K groups of sample defect identification results.

[0059] S630-8: Construct the K defect positioning sub-models based on the K groups of sample transfer image segmentation results and the K groups of sample defect identification results.

[0060] Specifically, the defective product positioning model is constructed based on a neural network, the input layer is used to input a to-be-identified image, the image segmentation layer is used to segment the to-be-identified image into a foreground layer and a background layer, the defect identification layer is used to identify a defective target product in the foreground layer according to a defect feature and perform positioning, and the output layer is used to output a result of the defect identification layer as defective product positioning information.

[0061] Specifically, the K defect positioning sub-models are respectively supervised trained by using corresponding target defect features as data sets. The image segmentation layer is supervised trained by using the sample transfer image set and the sample transfer image segmentation result as data sets, and preferably, an edge detection algorithm is used to construct the image segmentation layer. The edge detection algorithm includes a Canny edge detection algorithm, a Sobel operator, and the like.

[0062] Specifically, the training of the image segmentation layer is exemplarily performed by using the sample transfer image set as input training data, using the sample transfer image segmentation result as output training data, and performing associated storage to obtain a training data set. The training data set is divided into 7:3, wherein the 7 proportion is used for actually training a model, and the 3 proportion is used for verifying the output accuracy of the model. By using the 7 proportion of training data, a convolutional neural network is trained, when the deviation of the output of the model from the output identification information is less than or equal to a preset deviation, the accuracy of the model is verified by using the 3 proportion of training data, when the deviation of the output of a continuous set number of groups from the output identification information is less than or equal to the preset deviation, the model converges, and is stored as the image segmentation layer for segmentation of the background image and the foreground target product.

[0063] Further, the first I frame is input into the pre-constructed defective product positioning model to perform defective product positioning, and the defective product positioning information is obtained. Step S630 further includes:

[0064] S631: inputting the first I frame into the image segmentation layer of the defective product positioning model to obtain a first transfer image segmentation result, wherein the batch to-be-identified division image of the target batch product in the first transfer image segmentation result has a position identifier.

[0065] S632: inputting the real-time process step into the defect identification layer to activate the model, and obtaining a real-time defect positioning sub-model;

[0066] S633: inputting the first transfer image segmentation result into the real-time defect positioning sub-model to perform defective product positioning, and obtaining the defective product positioning information.

[0067] Specifically, according to the real-time process step, the applicable defect positioning sub-model is positioned. While ensuring the complete identification capability of the defect product positioning model, the first transfer image segmentation result is shunted, the defect positioning is refined, the identification efficiency and accuracy of the model are improved, and the technical effect of reducing the model complexity is realized.

[0068] S700: Based on the real-time production video information and the defect product positioning information, visual tracking of the defect product is performed to obtain a target defect positioning result.

[0069] Specifically, the defect product positioning information provides the initial position of the defect product, the real-time production video information provides the motion vector information of the defect product, and the initial position and the motion vector are combined to complete the visual tracking positioning of the defect product.

[0070] Further, the step S700 of performing visual tracking of the defect product based on the real-time production video information and the defect product positioning information to obtain a target defect positioning result includes:

[0071] S710: The defect product positioning information includes M defect product images, wherein each defect product image has a position identifier;

[0072] S720: Individual feature extraction is performed on the M defect product images to obtain M defect product global features;

[0073] S730: I-frame extraction is performed on each coding unit in the production video compression information to obtain I-frame information;

[0074] S740: Based on the M defect product global features, the I-frame information is matched to obtain M sets of defect motion trajectory nodes;

[0075] S750: Based on the M sets of defect motion trajectory nodes, motion trajectory prediction is performed to obtain M product trajectory prediction results;

[0076] S760: The M product trajectory prediction results constitute the target defect positioning result.

[0077] Specifically, the individual feature refers to the defect feature unique to the defect product, including defect type, size, position, number, etc. It is used to distinguish from other M-1 defect products and to match features with the I-frame information. Each I-frame obtains M motion trajectory nodes, and the M motion trajectory nodes correspond one-to-one to the M defect products. The I-frame information containing multiple I-frames is traversed to obtain multiple motion trajectory nodes, which correspond to the M defect products. For example, if the I-frame information contains N I-frames, each defect product corresponds to N motion trajectory nodes.

[0078] Specifically, the motion trajectory prediction refers to predicting the motion trend of the target according to the behavior characteristics of the moving target and combining the current position of the target. The method includes Kalman filtering, difference autoregressive moving average model, hidden Markov model, Gaussian mixture model, Bayesian network, BP neural network, RNN, etc. Exemplarily, the Kalman filtering method is used for the motion trajectory prediction. Through the motion trajectory prediction, the technical effect of accurately and quickly grabbing the limited product is realized.

[0079] Further, the visual tracking of the defective product based on the real-time production video information and the defective product positioning information obtains a target defect positioning result, and the step S700 further includes:

[0080] S770: calling an adjacent process step in the target process step set according to the real-time process step, to obtain a real-time adjacent step, wherein a product grabbing mechanical arm is arranged at an input end of the real-time adjacent step;

[0081] S780: sending the target defect positioning result to the real-time adjacent step;

[0082] S790: the real-time adjacent step activates the product grabbing mechanical arm based on the target defect positioning result to grab the defective product.

[0083] Specifically, the adjacent step refers to a process step whose input end is connected to the output end of the target process step. The grabbing mechanical arm is arranged at the input end of the real-time adjacent step, so that the defective product can be timely grabbed and removed before the next processing. Exemplarily, the arrangement position of the product grabbing mechanical arm is determined according to the actual situation, and preferably, the product grabbing mechanical arm is arranged at a position where the target product moves slowly and the movement changes little.

[0084] Specifically, the target defect positioning result refers to an information set including the position of the defective product and a corresponding time mark. Exemplarily, first, the product grabbing mechanical arm determines the interception position according to the grabbing speed, the current position of the product grabbing mechanical arm, and the target defect positioning result. Then, the product grabbing mechanical arm control system sends an operation signal, the product grabbing mechanical arm moves to the interception position, and the grabbing is completed. Through the analysis of the target defect positioning result, the technical effect of accurately intercepting and grabbing the defective product is realized.

[0085] In summary, the target positioning method of visual tracking provided by the application has the following technical effects:

[0086] This application breaks down the target process flow into process steps to obtain a set of target process steps. Then, visual inspection equipment is deployed based on this set, forming a visual inspection equipment set, with a one-to-one correspondence between the target process steps and the visual inspection equipment set. Next, the target batch of products is located based on the target process flow to obtain real-time process steps. Then, the corresponding real-time visual inspection equipment is activated according to the real-time process steps. Subsequently, images of the target batch of products are acquired using the real-time visual inspection equipment to obtain real-time production video information. Then, frames are extracted from the real-time production video information to locate defective products, obtaining defective product location information. Finally, visual tracking of defective products is performed based on the real-time production video information and the defective product location information to obtain the target defect location result. By deploying visual inspection equipment based on multiple target process steps to acquire and locate product images, this solves the technical problem faced by existing technologies where some workpieces already have defects in intermediate steps, requiring the remaining process flow to remove them, resulting in high production costs and resource waste. This achieves the technical effect of reducing production costs and minimizing resource waste.

[0087] Example 2

[0088] Based on the same concept as the target localization method for visual tracking in the above embodiment, such as Figure 3 As shown, this application also provides a visual tracking target localization system, the system comprising:

[0089] The process step acquisition module 11 is used to decompose the target process flow into process steps to obtain a set of target process steps, wherein the set of target process steps includes K target process steps.

[0090] The detection equipment deployment module 12 is used to deploy visual inspection equipment based on the target process step set to form a visual inspection equipment set, wherein the target process step set and the visual inspection equipment set have a one-to-one mapping relationship.

[0091] Product positioning module 13 is used to position target batch products based on the target process flow and obtain real-time process steps, wherein the real-time process steps belong to the target process step set.

[0092] Detection activation module 14, the detection activation module 14 is used to activate the real-time vision inspection equipment in the vision inspection equipment set according to the real-time process steps.

[0093] Image acquisition module 15, the image acquisition module 15 is used to acquire images of the target batch of products based on the real-time visual inspection device to obtain real-time production video information;

[0094] a defective product positioning module 16, configured to frame the real-time production video information, and perform defective product positioning based on real-time production frame images to obtain defective product positioning information;

[0095] a visual tracking module 17, configured to perform defective product visual tracking based on the real-time production video information and the defective product positioning information to obtain a target defective positioning result.

[0096] Further, the defective product positioning module 16 further comprises:

[0097] a defective feature set acquisition unit, configured to preset a defective feature recognition rule, perform defective feature recognition on the first defective image set based on the defective feature recognition rule, and obtain a first defective feature set;

[0098] a video compression unit, configured to perform compression processing on the real-time production video information to obtain production video compression information;

[0099] Further, the visual tracking module 17 further comprises a defective product grabbing unit, configured to activate the product grabbing mechanical arm based on the target defective positioning result to perform defective product grabbing.

[0100] It should be understood that the embodiments mentioned in the specification focus on the differences from other embodiments, and the specific embodiments in the foregoing embodiment one are also applicable to the visual tracking target positioning system in embodiment two, and for the sake of brevity of the specification, no further expansion is made here.

[0101] It should be understood that the embodiments disclosed in the present application and the above description can enable those skilled in the art to implement the present application. Meanwhile, the present application is not limited to the part of the embodiments mentioned above, and the embodiments mentioned in the present application are obvious modifications and variations, which also belong to the scope of the present application.

Claims

1. A method of target positioning for visual tracking, characterized by, The method comprises: process step splitting is performed on a target process flow to obtain a target process step set, wherein the target process step set comprises K target process steps; visual inspection equipment is arranged based on the target process step set to form a visual inspection equipment set, wherein the target process step set and the visual inspection equipment set have a one-to-one mapping relationship; target batch products are positioned based on the target process flow to obtain a real-time process step, wherein the real-time process step belongs to the target process step set; an active real-time visual inspection equipment is called in the visual inspection equipment set according to the real-time process step; image acquisition is performed on the target batch products based on the real-time visual inspection equipment to obtain real-time production video information; frame extraction is performed on the real-time production video information, and defect product positioning is performed based on real-time production frame images to obtain defect product positioning information; defect product visual tracking is performed based on the real-time production video information and the defect product positioning information to obtain a target defect positioning result; frame extraction is performed on the real-time production video information, and defect product positioning is performed based on real-time production frame images to obtain defect product positioning information, and the method further comprises: compression processing is performed on the real-time production video information to obtain production video compression information; I frame extraction is performed on a first encoding unit based on the production video compression information to obtain a first I frame; the first I frame is input into a pre-constructed defect product positioning model for defect product positioning to obtain the defect product positioning information The method further comprises: The defect product positioning model comprises an input layer, an image segmentation layer, a defect recognition layer and an output layer; The defect recognition layer comprises K defect positioning sub-models, wherein the K defect positioning sub-models are one-to-one mapped with the K target process steps; An interactive production log is obtained to obtain historical transfer image data of the same type of product as the target batch product to generate a sample transfer image set; The sample transfer image set is segmented and labeled into a conveyor belt background image and a product image to obtain a sample transfer image segmentation result; The image segmentation layer is constructed based on the sample transfer image set and the sample transfer image segmentation result; The sample transfer image segmentation result is split into K groups of sample transfer image segmentation results according to the K target process steps; K groups of sample defect identification results are obtained by identifying defects in the K groups of sample transfer image segmentation results according to the target defect feature set; The K defect positioning sub-models are constructed based on the K groups of sample transfer image segmentation results and the K groups of sample defect identification results.

2. The method of claim 1, wherein, Before the frame extraction is performed on the real-time production video information and the defect product positioning is performed based on the real-time production frame images to obtain the defect product positioning information, the method further comprises: a first target process step is called based on the target process step set; An interactive production log is obtained to obtain a first defect image set of the first target process step; Pre-set defect feature recognition rules, and perform defect feature recognition on the first defect image set based on the defect feature recognition rules to obtain a first defect feature set; By analogy, a target defect feature set of the target process step set is obtained.

3. The method of claim 1, wherein, The first I frame is input into a pre-constructed defect product positioning model to perform defect product positioning, and defect product positioning information is obtained. The method further includes: The first I frame is input into the image segmentation layer of the defect product positioning model to obtain a first transfer image segmentation result, wherein the batch of to-be-identified division images of the target batch product in the first transfer image segmentation result have position labels. The real-time process step is input into the defect recognition layer to activate the real-time defect positioning sub-model, and a real-time defect positioning sub-model is obtained. The first transfer image segmentation result is input into the real-time defect positioning sub-model to perform defect product positioning, and the defect product positioning information is obtained.

4. The method of claim 1, wherein, Based on the real-time production video information and the defect product positioning information, visual tracking of the defect product is performed to obtain a target defect positioning result. The method further includes: The defect product positioning information includes M defect product images, wherein each defect product image has a position label. Individual features of the M defect product images are extracted to obtain M defect product global features. I frames are extracted from each coding unit in the production video compression information to obtain I frame information. The M defect product global features are matched with the I frame information to obtain M groups of defect motion trajectory nodes. Motion trajectory prediction is performed based on the M groups of defect motion trajectory nodes to obtain M product trajectory prediction results. The M product trajectory prediction results constitute the target defect positioning result.

5. The method of claim 1, wherein, The method further includes: Adjacent process steps are called in the target process step set according to the real-time process step, and a real-time adjacent step is obtained, wherein a product grabbing mechanical arm is arranged at the input end of the real-time adjacent step. The target defect positioning result is sent to the real-time adjacent step. The real-time adjacent step activates the product grabbing mechanical arm based on the target defect positioning result to grab the defect product.

6. A vision tracking target positioning system for implementing the method of claim 1, characterized by The system includes: A process step acquisition module is configured to split a target process into process steps to obtain a target process step set, wherein the target process step set includes K target process steps. A detection device arrangement module is configured to arrange visual detection devices based on the target process step set to form a visual detection device set, wherein the target process step set and the visual detection device set have a one-to-one mapping relationship. A product positioning module is configured to position a target batch product based on the target process to obtain a real-time process step, wherein the real-time process step belongs to the target process step set. A detection activation module is configured to activate a real-time visual detection device based on the real-time process step in the visual detection device set. An image acquisition module is configured to acquire images of the target batch of products based on the real-time visual inspection equipment, and obtain real-time production video information; A defective product positioning module is configured to frame the real-time production video information, and position defective products based on real-time production frame images, and obtain defective product positioning information; A visual tracking module is configured to visually track the defective products based on the real-time production video information and the defective product positioning information, and obtain a target defective positioning result.

Images