Camera automatic identification tracking method and system based on machine vision

Abstract

The invention discloses a camera automatic identification tracking method and system based on machine vision, and relates to the technical field of camera identification tracking, and the method comprises the following steps: configuring and calibrating a high-definition camera, obtaining the image and video data of a target environment through the high-definition camera, and storing the image and video data in a database; the method comprises the following steps of: preprocessing image and video data acquired by a high-definition camera, storing the preprocessed image and video data according to time, place and category, deleting fuzzy image and video data, identifying a target object in the image and video data by using a target detection algorithm, and identifying and tracking the target object; through the target detection algorithm and the target tracking model, the target object of the image and video data can be accurately identified and tracked in real time, the monitoring efficiency is greatly improved, misjudgment and missed judgment caused by human factors are reduced, meanwhile, the labor cost is reduced, the monitoring work is more efficient and reliable, and the user experience is improved. And a powerful guarantee is provided for public safety.

Description

Technical Field

[0001] The present invention relates to the technical field of camera recognition and tracking, and in particular to a camera automatic recognition and tracking method and system based on machine vision. Background Art

[0002] With the rapid development of computer vision technology, machine vision, as one of its core branches, has made remarkable progress in image recognition, target detection, motion tracking and other fields. By simulating the human visual system, machine vision can automatically extract useful information from images or videos, and process and analyze them. With the continuous development and popularization of artificial intelligence technology, camera automatic recognition and tracking technology will be applied and developed in more fields, and widely used in security monitoring, intelligent transportation, industrial automation and other fields.

[0003] In the existing technology, the camera is easily interfered by various environmental factors during recognition and tracking, which makes it difficult for the camera to recognize and the image is blurred. In addition, when the target object or environment changes dynamically, it is difficult to predict the dynamic changes. Therefore, how to weaken the interference of environmental factors and enhance the dynamic recognition and tracking capabilities is the problem we need to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a camera automatic identification and tracking method and system based on machine vision to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: In a first aspect, a camera automatic recognition and tracking method based on machine vision comprises the following steps: step 1, configuring and calibrating a high-definition camera, and using the high-definition camera to obtain image and video data of a target environment; Step 2, pre-processing the image and video data acquired by the high-definition camera, storing the pre-processed image and video data according to time, location, and category, and deleting blurred image and video data; Step 3: Use the target detection algorithm to identify the target object in the image and video data, identify and track the target object, and extract the features of the target object for classification. At the same time, use the features of the target object and combine the image and video data to build a target tracking model, and then predict the position and state of the target object in subsequent frames, and evaluate the prediction results to ensure that the target object can still be accurately tracked when the environment changes dynamically; Step 4: Obtain environmental factors such as lighting conditions and background changes, dynamically adjust the high-definition camera, and set up an anomaly detection mechanism to analyze abnormal situations during the identification and tracking process, thereby optimizing the tracking strategy; Step 5: Visualize the results of recognition and tracking, and provide feedback and adjust the image and video data based on the results of the visualization and actual needs.

[0006] A further improvement of the technical solution of the present invention is that the process of acquiring the image and video data of the target environment includes: Identify the monitoring area and target requirements, select appropriate HD cameras, and prepare installation materials and tools to install the HD cameras in the predetermined locations to cover the required monitoring area; Adjust the angle, focal length and image parameters of the HD camera, test the image effect of the camera under different lighting conditions, and make fine adjustments based on the test results; Set the recording parameters in the HD camera's setting interface, including recording time, recording resolution, storage location, and back them up regularly.

[0007] A further improvement of the technical solution of the present invention is that the process of preprocessing the image and video data acquired by the high-definition camera includes: Set pixel thresholds and calculate the standard deviation of image grayscale values ​​to identify image and video data; Comparing the correlation between the pixel frame rate of the image and video data and the pixel threshold, if the pixel frame rate of the image and video data exceeds the set pixel threshold, the image and video data are determined to be invalid; if the pixel frame rate of the image and video data does not exceed the set pixel threshold, the image and video data are determined to be valid; Preprocess the valid image and video data, including denoising, contrast adjustment and size normalization. Gaussian filtering is used to reduce random noise in the image, and linear contrast stretching is used to enhance the visual effect of the image. The image and video data are then scaled and cropped and adjusted to a uniform size.

[0008] A further improvement of the technical solution of the present invention is that the process of evaluating the prediction results is: Prepare a training data set to train the target detection model, and use the trained target detection model to detect the target object, wherein the training data set includes positive samples, wherein the positive samples are images containing the target object, and the negative samples are images not containing the target object; Use KCF (Kernelized Correlation Filters) tracking algorithm to associate target objects, extract features from target objects, and then use the extracted features to classify target objects; A target tracking model is established using the extracted features of the target object, and the target tracking model is used to predict the position of the target object; Update the state estimate based on the predicted position of the target object and use the intersection-over-union ratio to evaluate the tracking accuracy of the target tracking model. The target detection model and target tracking model are continuously optimized based on the evaluation results. Through the target detection algorithm and target tracking model, the target objects of image and video data can be identified and tracked in real time and accurately, which greatly improves the monitoring efficiency and reduces the misjudgment and missed judgment caused by human factors. At the same time, it reduces labor costs, makes monitoring work more efficient and reliable, and provides a strong guarantee for public safety.

[0009] A further improvement of the technical solution of the present invention is that the calculation formula of the intersection-over-union ratio is: ; ; ; in, is the intersection-and-union ratio, is represented as the intersection area, is represented as the union region, is the predicted bounding box, Expressed as , is the ground-truth bounding box, Expressed as , and is the coordinate of the center point of the predicted bounding box, and is the coordinate of the center point of the real bounding box, and is the width and height of the predicted bounding box, and are the width and height of the true bounding box.

[0010] A further improvement of the technical solution of the present invention is that the step 4 comprises: Use statistical methods to detect light intensity and Gaussian mixture model to build background model to extract foreground targets, and adjust the exposure time, gain and white balance of the camera according to the light intensity. The statistical method is the histogram of the image. According to the abnormal behavior characteristics of speed anomaly, shape change and motion pattern deviation, an anomaly detection mechanism is constructed and associated with the background model, and an anomaly detection threshold is set. The anomaly of the anomaly detection threshold is evaluated, and the anomaly is fed back to the tracking model for adjustment.

[0011] A further improvement of the technical solution of the present invention is that the calculation formula of the abnormality detection threshold is: ; ; in, is the anomaly detection threshold, is the mean anomaly score, is the standard deviation of the anomaly score, is the confidence level, is the anomaly score, is the mean of the feature, It is The mean of the features, It is The standard deviation of the feature, is the number of features.

[0012] A further improvement of the technical solution of the present invention is that the step 5 comprises: Extract the location, size, category, and tracking status of the target object, and display the target object's bounding box, tracking trajectory, and category label on the visualization interface; Mark the category and tracking ID of the target object in the visual interface, and use different colors to distinguish different tracking states; Observe the visual display of tracking loss, analyze the recognition accuracy and tracking stability of the target object, adjust the image and video data, conduct in-depth analysis and mining of the data generated during the tracking process through real-time tracking of the target object, analyze the target's movement trajectory and behavior patterns, discover potential safety hazards and regular events, provide strong data support, and improve the overall management level and emergency response capabilities.

[0013] In the second aspect, a camera automatic recognition and tracking system based on machine vision is used to implement a camera automatic recognition and tracking method based on machine vision, including a monitoring center, and the monitoring center is communicatively connected with a data acquisition module, a preprocessing module, a target recognition module, a target tracking module, an anomaly detection module, a performance evaluation module, a visualization display module and a user interaction module.

[0014] A further improvement of the technical solution of the present invention is that: the data acquisition module is used to obtain the real-time image and video data stream of the camera; The preprocessing module is used to preprocess the acquired image; The target recognition module recognizes the target object in the image and video data according to the target detection algorithm, and determines its bounding box coordinates, bounding box size and category; The target tracking module tracks the target object in the continuous frames according to the tracking algorithm and records its tracking ID and tracking status; The anomaly detection module is used to analyze anomalies during the tracking process; The performance evaluation module is used to evaluate the performance of the tracking model. By integrating the anomaly detection module and the performance evaluation module, the module monitors the problems that occur during the tracking process in real time, provides timely feedback, and triggers the corresponding adjustment mechanism. At the same time, the performance of the tracking model is evaluated through quantitative evaluation indicators. When facing a complex and changing environment, the module automatically adjusts the parameters to improve the adaptability and robustness of the system, so that the target tracking model can maintain high accuracy and stability under different environmental conditions; The visualization display module is used to display the target object's bounding box, tracking trajectory, category label and other information on the visualization interface, mark the target object's category and tracking ID, and use different colors to distinguish the tracking status.

[0015] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art: The present invention provides a method and system for automatic camera identification and tracking based on machine vision. Through target detection algorithm and target tracking model, the target object of image and video data can be identified and tracked in real time and accurately, which greatly improves the monitoring efficiency, reduces the misjudgment and missed judgment caused by human factors, and reduces the labor cost, making the monitoring work more efficient and reliable, and providing a strong guarantee for public safety. The present invention provides a method and system for automatic camera identification and tracking based on machine vision. Through the integration of abnormal detection module and performance evaluation module, the problems occurring in the tracking process are monitored in real time, and feedback is provided in time to trigger the corresponding adjustment mechanism. At the same time, the performance of the tracking model is evaluated through quantitative evaluation indicators. When facing a complex and changeable environment, the parameters are automatically adjusted to improve the adaptability and robustness of the system, so that the target tracking model can maintain high accuracy and stability under different environmental conditions. The present invention provides a method and system for automatic camera identification and tracking based on machine vision. Through real-time tracking of the target object, the data generated during the tracking process are deeply analyzed and mined, the target motion trajectory and behavior pattern are analyzed, potential safety hazards and regular events are discovered, strong data support is provided, and the overall management level and emergency response capability are improved. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1 is a flow chart of the method of the present invention; Figure 2 A flow chart for evaluating the prediction results of the present invention; Figure 3It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1, as Figure 1 As shown, the present invention provides a camera automatic recognition and tracking method based on machine vision, comprising the following steps: Step 1: Configure and calibrate a high-definition camera, and use the high-definition camera to obtain image and video data of the target environment.

[0018] Step 2, pre-process the image and video data obtained by the high-definition camera, store the pre-processed image and video data according to time, location, and category, and delete the blurred image and video data.

[0019] Among them, preprocessing includes denoising, contrast adjustment, edge detection and size standardization preprocessing of images and video data obtained by high-definition cameras to improve image quality and facilitate subsequent feature extraction and recognition.

[0020] Step 3: Use the target detection algorithm to identify the target object in the image and video data, identify and track the target object, extract the features of the target object for classification, and use the features of the target object in combination with the image and video data to build a target tracking model, thereby predicting the position and state of the target object in subsequent frames and evaluating the prediction results. This ensures that the target object can be accurately tracked when the environment changes dynamically.

[0021] It should be noted that the target detection algorithm is one of YOLO, Faster R-CNN, and SSD, which is used to detect targets in images; the deep learning model is ResNet or VGG, which identifies the target and determines its category; the feature information of the target object includes shape, color, texture, etc.; the features will be used for subsequent tracking and recognition, and the tracking algorithm is used to associate targets in consecutive frames, predict their motion trajectories, and predict the position and state of the target in the next frame or subsequent frames.

[0022] Step 4: Obtain environmental factors such as lighting conditions and background changes, dynamically adjust the high-definition camera, and set up an anomaly detection mechanism to analyze abnormal situations during the identification and tracking process, thereby optimizing the tracking strategy.

[0023] Step 5: Visualize the results of recognition and tracking, and provide feedback and adjust the image and video data based on the results of the visualization and actual needs.

[0024] The automatic recognition and tracking method of a camera based on machine vision of the present invention can, through a target detection algorithm and a target tracking model, accurately recognize and track target objects in image and video data in real time, greatly improve monitoring efficiency, reduce misjudgments and missed judgments caused by human factors, and at the same time, reduce labor costs, making monitoring work more efficient and reliable, and providing a strong guarantee for public safety.

[0025] According to one embodiment of the present invention, the process of acquiring image and video data of the target environment includes: clarifying the monitoring area and target requirements, selecting a suitable high-definition camera, and preparing installation materials and tools to install the high-definition camera to a predetermined position to cover the required monitoring area, adjusting the angle, focal length and image parameters of the high-definition camera, testing the image effect of the camera under different lighting conditions, and fine-tuning according to the test results, setting the recording parameters in the setting interface of the high-definition camera, including recording time, recording resolution, storage location, and backing it up regularly.

[0026] According to one embodiment of the present invention, the process of preprocessing image and video data acquired by a high-definition camera includes: setting a pixel threshold and calculating the standard deviation of the image grayscale value to identify the image and video data, comparing the correlation between the pixel frame rate of the image and video data and the pixel threshold, if the pixel frame rate of the image and video data exceeds the set pixel threshold, then the image and video data are determined to be invalid, if the pixel frame rate of the image and video data does not exceed the set pixel threshold, then the image and video data are determined to be valid, and the valid image and video data are preprocessed, wherein the preprocessing includes denoising, contrast adjustment and size standardization, and Gaussian filtering is used to reduce random noise in the image, linear contrast stretching is used to enhance the visual effect of the image, and then the image and video data are scaled and cropped, and adjusted to a uniform size.

[0027] like Figure 2As shown, according to one embodiment of the present invention, the process of evaluating the prediction result is: preparing a training data set to train a target detection model, and using the trained target detection model to detect the target object, wherein the training data set includes positive samples and negative samples, the positive sample is an image containing the target object, and the negative sample is an image not containing the target object, and at the same time, the KCF tracking algorithm is used to associate the target object, extract features of the target object, and then classify the target object using the extracted features, and use the extracted features of the target object to build a target tracking model, and use the target tracking model to predict the position of the target object, update the state estimate according to the predicted position of the target object, and use the intersection and union ratio to evaluate the tracking accuracy of the target tracking model, and continuously optimize the target detection model and the target tracking model according to the evaluation results. Through the target detection algorithm and the target tracking model, the target objects of the image and video data can be identified and tracked in real time and accurately, which greatly improves the monitoring efficiency, reduces the misjudgment and missed judgment caused by human factors, and at the same time, reduces the labor cost, makes the monitoring work more efficient and reliable, and provides a strong guarantee for public safety.

[0028] Furthermore, the calculation formula of the intersection-over-union ratio is: ; ; ; in, is the intersection-and-union ratio, is represented as the intersection area, is represented as the union region, is the predicted bounding box, Expressed as , is the ground-truth bounding box, Expressed as , and is the coordinate of the center point of the predicted bounding box, and is the coordinate of the center point of the real bounding box, and is the width and height of the predicted bounding box, and are the width and height of the true bounding box.

[0029] According to an embodiment of the present invention, the above step 4 includes: using a statistical method to detect the light intensity, and using a Gaussian mixture model to establish a background model to extract the foreground target, and adjusting the exposure time, gain and white balance of the camera according to the light intensity, the statistical method is the histogram of the image, and at the same time, according to the abnormal behavior characteristics of speed abnormality, shape change and motion mode deviation, an abnormal detection mechanism is constructed and associated with the background model, and an abnormal detection threshold is set, and the abnormal situation of the abnormal detection threshold is evaluated and the abnormal situation is fed back to the tracking model and adjusted. By integrating the abnormal detection module and the performance evaluation module, the problems occurring in the tracking process are monitored in real time, and feedback is provided in a timely manner to trigger the corresponding adjustment mechanism. At the same time, the performance of the tracking model is evaluated through quantitative evaluation indicators. When facing a complex and changing environment, the parameters are automatically adjusted to improve the adaptability and robustness of the system, so that the target tracking model can maintain high accuracy and stability under different environmental conditions.

[0030] Furthermore, the calculation formula of the anomaly detection threshold is: ; ; in, is the anomaly detection threshold, is the mean anomaly score, is the standard deviation of the anomaly score, is the confidence level, is the anomaly score, is the mean of the feature, It is The mean of the features, It is The standard deviation of the feature, is the number of features.

[0031] According to an embodiment of the present invention, the above step 5 includes: extracting the position, size, category and tracking status of the target object, and displaying the bounding box, tracking trajectory and category label of the target object to the visualization interface, marking the category and tracking ID of the target object in the visualization interface, and using different colors to distinguish different tracking states, observing the tracking loss situation displayed visually, analyzing the recognition accuracy and tracking stability of the target object, and adjusting the image and video data. Through real-time tracking of the target object, the data generated during the tracking process is deeply analyzed and mined, the target movement trajectory and behavior pattern are analyzed, potential safety hazards and regular events are discovered, strong data support is provided, and the overall management level and emergency response capabilities are improved.

[0032] Embodiment 2, as Figure 3As shown, on the basis of Example 1, the present invention also provides a camera automatic identification and tracking system based on machine vision, including a monitoring center, the monitoring center is communicatively connected with a data acquisition module, a preprocessing module, a target identification module, a target tracking module, an anomaly detection module, a performance evaluation module, a visualization display module and a user interaction module, wherein each module is electrically connected.

[0033] Data acquisition module, used to obtain real-time images and video data streams from the camera; A preprocessing module, used for preprocessing the acquired image; The target recognition module identifies the target objects in the image and video data according to the target detection algorithm, and determines their bounding box coordinates, bounding box size and category; The target tracking module tracks the target object in consecutive frames according to the tracking algorithm and records its tracking ID and tracking status; Anomaly detection module, used to analyze anomalies during tracking; Performance evaluation module, used to evaluate the performance of the tracking model; The visualization display module is used to display the target object’s bounding box, tracking trajectory, category label and other information on the visualization interface, mark the target object’s category and tracking ID, and use different colors to distinguish the tracking status.

[0034] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A camera automatic recognition and tracking method based on machine vision, characterized in that: The following steps are involved: Step 1: configure and calibrate a high-definition camera, and use the high-definition camera to obtain image and video data of the target environment; Step 2, pre-processing the image and video data acquired by the high-definition camera, storing the pre-processed image and video data according to time, location, and category, and deleting blurred image and video data; Step 3: Use the target detection algorithm to identify the target object in the image and video data, identify and track the target object, and extract the features of the target object for classification. At the same time, use the features of the target object and combine the image and video data to build a target tracking model, and then predict the position and state of the target object in subsequent frames, and evaluate the prediction results; Step 4: Obtain environmental factors such as lighting conditions and background changes, dynamically adjust the high-definition camera, and set up an anomaly detection mechanism to analyze abnormal situations during the identification and tracking process, thereby optimizing the tracking strategy; Step 5: Visualize the results of recognition and tracking, and provide feedback and adjust the image and video data based on the results of the visualization and actual needs.

2. The camera automatic identification and tracking method based on machine vision according to claim 1, characterized in that: The process of acquiring image and video data of the target environment includes: Identify the monitoring area and target requirements, select HD cameras, and install them in the intended locations to cover the required monitoring area; Adjust the angle, focal length and image parameters of the HD camera, test the image effect of the camera under different lighting conditions, and make fine adjustments based on the test results; Set the recording parameters in the HD camera's setting interface, including recording time, recording resolution, storage location, and back them up regularly.

3. The camera automatic recognition and tracking method based on machine vision according to claim 1, characterized in that: The process of preprocessing the image and video data acquired by the high-definition camera includes: Set pixel thresholds and calculate the standard deviation of image grayscale values ​​to identify image and video data; Comparing the correlation between the pixel frame rate of the image and video data and the pixel threshold, if the pixel frame rate of the image and video data exceeds the set pixel threshold, the image and video data are determined to be invalid; if the pixel frame rate of the image and video data does not exceed the set pixel threshold, the image and video data are determined to be valid; Preprocess valid image and video data, including denoising, contrast adjustment and size normalization.

4. The camera automatic recognition and tracking method based on machine vision according to claim 1, characterized in that: The process of evaluating the prediction results is as follows: Prepare a training data set to train the target detection model, and use the trained target detection model to detect the target object, where the training data set includes positive samples and negative samples; Use the KCF tracking algorithm to associate the target object, extract features from the target object, and then use the extracted features to classify the target object; A target tracking model is established using the extracted features of the target object, and the target tracking model is used to predict the position of the target object; Update the state estimate based on the predicted position of the target object and use the intersection-over-union ratio to evaluate the tracking accuracy of the target tracking model. Continuously optimize the target detection model and target tracking model based on the evaluation results.

5. The camera automatic identification and tracking method based on machine vision according to claim 4 is characterized in that: The calculation formula of the intersection-over-union ratio is: ； ； ； in, is the intersection-and-union ratio, is represented as the intersection area, is the ground-truth bounding box, is represented as the union region, is the predicted bounding box, Expressed as , is the ground-truth bounding box, Expressed as , and is the coordinate of the center point of the predicted bounding box, and is the coordinate of the center point of the real bounding box, and is the width and height of the predicted bounding box, and are the width and height of the ground-truth bounding box.

6. The camera automatic identification and tracking method based on machine vision according to claim 1, characterized in that: The step 4 comprises: Use statistical methods to detect light intensity and Gaussian mixture model to build background model to extract foreground targets, and adjust the camera exposure time, gain and white balance according to the light intensity; According to the abnormal behavior characteristics of speed anomaly, shape change and motion pattern deviation, an anomaly detection mechanism is constructed and associated with the background model, and an anomaly detection threshold is set. The anomaly of the anomaly detection threshold is evaluated, and the anomaly is fed back to the tracking model for adjustment.

7. The camera automatic identification and tracking method based on machine vision according to claim 6, characterized in that: The calculation formula of the anomaly detection threshold is: ； ； in, is the anomaly detection threshold, is the mean anomaly score, is the standard deviation of the anomaly score, is the confidence level, is the anomaly score, is the mean of the feature, It is The mean of the features, It is The standard deviation of the feature, is the number of features.

8. The camera automatic identification and tracking method based on machine vision according to claim 1, characterized in that: The step 5 comprises: Extract the location, size, category, and tracking status of the target object, and display the target object's bounding box, tracking trajectory, and category label on the visualization interface; Mark the category and tracking ID of the target object in the visual interface, and use different colors to distinguish different tracking states; Observe the visual display of tracking loss, analyze the recognition accuracy and tracking stability of the target object, and make adjustments to the image and video data.

9. A camera automatic identification and tracking system based on machine vision, used to implement the camera automatic identification and tracking method based on machine vision according to any one of claims 1 to 8, comprising a monitoring center, characterized in that: The monitoring center is communicatively connected with a data acquisition module, a preprocessing module, a target recognition module, a target tracking module, an anomaly detection module, a performance evaluation module, a visualization display module and a user interaction module.

10. The camera automatic identification and tracking system based on machine vision according to claim 9, characterized in that: The data acquisition module is used to obtain real-time images and video data streams from the camera; The preprocessing module is used to preprocess the acquired image; The target recognition module recognizes the target object in the image and video data according to the target detection algorithm, and determines its bounding box coordinates, bounding box size and category; The target tracking module tracks the target object in the continuous frames according to the tracking algorithm and records its tracking ID and tracking status; The anomaly detection module is used to analyze anomalies during the tracking process; The performance evaluation module is used to evaluate the performance of the tracking model; The visualization display module is used to display the target object's bounding box, tracking trajectory, category label and other information on the visualization interface, mark the target object's category and tracking ID, and use different colors to distinguish the tracking status.

Images