Pan-tilt-zoom tracking method and system based on mobile detection and neural network

Through the gimbal tracking method based on mobile detection and neural network, combined with convolutional neural network and light-sensitive diagnostic algorithm, the problem of poor accuracy and real-time performance in gimbal tracking technology is solved, and efficient character recognition and tracking is achieved , improves the accuracy and efficiency of tracking through compensation.

CN119006527AActive Publication Date: 2024-11-22SHANGHAI FULLHAN MICROELECTRONICS
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411252013.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-22
Estimated Expiration
2044-09-09

AI Technical Summary

Technical Problem

The existing gimbal tracking technology has poor accuracy and real-time performance in the security field. The calculation volume of deep learning algorithms is large, while the real-time performance of traditional image processing methods is limited, making it difficult to achieve efficient tracking.

Method used

Using a gimbal tracking method based on motion detection and neural network, the preprocessing and feature extraction of image frame sequences is realized through motion recognition model, character extraction model and rotation angle model, combined with convolutional neural network and light-sensitive diagnostic algorithm. and real-time tracking, using matching algorithms and compensation methods to optimize tracking accuracy and efficiency.

Benefits of technology

It improves the accuracy of character recognition and the real-time tracking, solves the problems of large amount of computing and poor real-time performance brought about by deep learning methods, and improves the accuracy and efficiency of tracking through central area, reverse rotation and edge compensation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006527A_ABST
    Figure CN119006527A_ABST
Patent Text Reader

Abstract

The invention relates to a PTZ tracking method and system based on mobile detection and a neural network, and belongs to the technical field of trajectory tracking. The method comprises the following steps: calculating to obtain a motion frame set through a motion identification model according to a time sequence image frame sequence, and calculating to obtain an identification frame set through an identification model according to the time sequence image frame sequence; according to the motion frame set and the recognition frame set, a target motion frame set is obtained through calculation of a rejection model, and a motion macro block set and a target person are obtained through calculation of a motion extraction model and a person extraction model; obtaining a reference coordinate set, obtaining a target coordinate, and traversing the reference coordinate set according to the target coordinate to obtain a target macro block group; and acquiring an actually measured macro block group set, calculating to obtain a shift macro block group, calculating to obtain a rotation angle through a rotation angle model according to the target macro block group and the shift macro block group, and controlling the holder machine to perform tracking by the intelligent terminal according to the rotation angle. And people in the warning area can be tracked by the holder machine.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of trajectory tracking, and in particular relates to a pan / tilt machine tracking method and system based on motion detection and neural network. Background Art

[0002] Tracking technology is an important task for sensors and machine vision technology, especially in the fields of home and security, which are becoming more and more popular and gradually playing an important role. On the one hand, with the continuous advancement of IoT technology and the expansion of its application areas, the application scenarios of tracking technology will continue to expand; on the other hand, with the continuous maturity of artificial intelligence and security fields and the continuous upgrading of consumer demand, the tracking technology used by PTZ machines will play a more important role.

[0003] With the continuous development of artificial intelligence, relevant tracking technologies in the security field have made significant progress, but they still face challenges in practical application. For example, although the algorithm based on deep learning has high accuracy, it has a large amount of calculation and poor real-time performance; while the traditional image processing method has good real-time performance but limited accuracy. Therefore, there is still room for improvement in the tracking technology of PTZ machines. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a pan / tilt machine tracking method based on motion detection and neural network.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] S1: Obtain statistical image data of motion detection through a PTZ machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model;

[0007] S2: Calculate a target motion frame set by using a elimination model according to the motion frame set and the identification frame set, calculate a motion macroblock set by using a motion extraction model according to the target motion frame set, and calculate a target person by using a person extraction model according to the target motion frame set;

[0008] S3: acquiring a reference coordinate set according to the motion macroblock set, acquiring a target coordinate according to the target person, and traversing and screening the reference coordinate set according to the target coordinate to obtain a target macroblock set;

[0009] S4: real-time image frames are acquired through the gimbal, a measured macroblock group set is calculated based on the real-time image frames through the motion extraction model, a shifted macroblock group is calculated based on the target macroblock group and the measured macroblock group set through a matching algorithm, a rotation angle is calculated based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal for tracking.

[0010] As a preferred technical solution of the present invention, the motion recognition model preprocesses the time-series image frame sequence in sequence to obtain a preprocessed image sequence, and then grayscales the preprocessed image sequence to obtain a grayscale image sequence, calculates the grayscale value of each grayscale image according to the grayscale image sequence and the color channel weight, calculates the grayscale difference between two consecutive frames of the grayscale image according to the grayscale value, compares the grayscale difference with a preset threshold to obtain a change frame, and gathers each of the change frames to obtain the motion frame set.

[0011] Specifically, the recognition model is implemented based on a convolutional neural network and is used to determine whether a person appears in the temporal image frame sequence. The specific calculation steps are as follows:

[0012] S301: Preprocessing the time-series image frame sequence to obtain a preprocessed image sequence, wherein the preprocessing includes image grayscale conversion, denoising, and image enhancement;

[0013] S302: extracting features from the preprocessed image sequence to obtain feature data, wherein the feature data includes shape features and texture features;

[0014] S303: performing dimensionality reduction processing according to the feature data to obtain dimensionality reduction features, wherein the dimensionality reduction processing includes average normalization, standardization, zero component analysis, and dimensionality reduction calculation;

[0015] S304: Obtain a recognition result by calculating a classifier according to the dimension reduction feature, and label the temporal image frame sequence according to the recognition result to obtain the recognition frame set.

[0016] Specifically, the elimination model first obtains a misjudgment result according to the motion frame set through a light-sensitive diagnosis algorithm, eliminates the motion frame set according to the misjudgment result to obtain a purified frame set, and traverses and filters the purified frame set according to the timestamp of the identified frame set to obtain the target motion frame set. The calculation formula of the light-sensitive diagnosis algorithm is:

[0017]

[0018] Where x, y are the pixel coordinates, A is the image area, S(x, y, t) is the grayscale value of the pixel (x, y) at time t, a is the time difference, c is the adjustment coefficient, b is the number of pixels in the detection area, and N is the preset threshold.

[0019] Specifically, the motion extraction model obtains a grayscale frame set by graying the target frame set, obtains a divided frame set by performing macroblock division on the grayscale frame set according to a preset macroblock width, calculates the grayscale value difference between two consecutive frames according to the divided frame set, compares the grayscale value difference with a preset threshold value to obtain a motion macroblock determination result, labels the macroblocks according to the motion macroblock determination result to obtain motion macroblocks, and aggregates adjacent motion macroblocks to obtain the motion macroblock group set.

[0020] Specifically, the character extraction model is implemented based on the recognition model. The character recognition result set is calculated by the recognition model according to the target motion frame set, the coordinate domain of the character recognition result set is obtained, the coordinate domain is compared with a preset coordinate frame threshold to obtain a comparison result, and the character recognition result set is screened according to the comparison result to obtain the target person.

[0021] Specifically, the rotation angle includes a horizontal rotation angle and a vertical rotation angle. The rotation angle model determines the movement direction of the PTZ machine by calculating the coordinate difference between the target macroblock group and the measured macroblock group, and obtains the horizontal rotation angle and the vertical rotation angle according to the coordinates of the measured macroblock group. The horizontal rotation angle calculation formula is:

[0022]

[0023] Among them, θ x is the horizontal rotation angle, x is the measured macroblock group horizontal coordinate, w is the picture width, and θx0 is the horizontal field angle;

[0024] The vertical rotation angle calculation formula is:

[0025]

[0026] Among them, θ y is the horizontal rotation angle, y is the vertical coordinate of the measured macroblock group, h is the picture height, and θy0 is the vertical field angle.

[0027] A pan-tilt machine tracking method based on motion detection and neural network and a pan-tilt machine tracking system based on motion detection and neural network, comprising: an image acquisition module, an extraction module, a comparison module, and a tracking module;

[0028] The image acquisition module is used to acquire statistical image data of motion detection through a pan-tilt machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model;

[0029] The extraction module is used to calculate a target motion frame set according to the motion frame set and the identification frame set through a elimination model, calculate a motion macroblock set according to the target motion frame set through a motion extraction model, and calculate a target person according to the target motion frame set through a person extraction model;

[0030] The comparison module is used to obtain a reference coordinate set according to the motion macroblock set, obtain a target coordinate according to the target person, and traverse and filter the reference coordinate set according to the target coordinate to obtain a target macroblock set;

[0031] The tracking module is used to obtain real-time image frames through the gimbal machine, calculate the measured macroblock group set based on the real-time image frames through the motion extraction model, calculate the shifted macroblock group based on the target macroblock group and the measured macroblock group through a matching algorithm, calculate the rotation angle based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal machine for tracking.

[0032] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the pan / tilt machine tracking method based on motion detection and neural network as described above is implemented.

[0033] A storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the pan-tilt machine tracking method based on motion detection and neural network as described above when executed by a computer processor.

[0034] The beneficial effects of the present invention are:

[0035] (1) In the tracking and recognition stage, a recognition model based on a convolutional neural network is used to perform person recognition on the collected image data to obtain image frames containing people, thereby improving the accuracy of person recognition.

[0036] (2) In the tracking stage, the target person is obtained according to the person extraction model. The reference coordinate set is traversed according to the coordinates of the target person to obtain the target macroblock group. The target macroblock group is then tracked in real time through motion vector search, which solves the problems of poor real-time performance and large amount of computation caused by the use of deep learning methods in the tracking stage.

[0037] (3) During tracking, the combination of central area compensation, reverse rotation compensation, and edge compensation can more accurately adapt to the characteristics of motor rotation and improve tracking accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0039] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0040] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0041] See also Figure 1 , a PTZ tracking method based on motion detection and neural network

[0042] S1: Obtain statistical image data of motion detection through a PTZ machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model; the recognition frame set is an image frame sequence containing a person recognition result;

[0043] S2: Calculate a target motion frame set by using a elimination model based on the motion frame set and the identification frame set, calculate a motion macroblock set by using a motion extraction model based on the target motion frame set, and calculate a target person by using a person extraction model based on the target motion frame set; the motion macroblock set is composed of adjacent motion macroblocks and is used to represent a moving object;

[0044] S3: acquiring a reference coordinate set according to the motion macroblock group set, acquiring a target coordinate according to the target person, and traversing and screening the reference coordinate set according to the target coordinate to obtain a target macroblock group; the reference coordinate set is composed of the center point coordinates of each motion macroblock group, and the target coordinates are the center point coordinates of the target person;

[0045] S4: real-time image frames are acquired through the gimbal, a measured macroblock group set is calculated based on the real-time image frames through the motion extraction model, a shifted macroblock group is calculated based on the target macroblock group and the measured macroblock group set through a matching algorithm, a rotation angle is calculated based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal for tracking.

[0046] In this embodiment, the matching algorithm searches the measured macroblock group set through the motion vector according to the target macroblock group in the previous frame to obtain the shifted macroblock group in the next frame. The calculation formula of the matching algorithm is:

[0047]

[0048] Among them, (x, y) is the coordinate of the target macroblock group, MCF is the matching cost function, A is the allowed motion vector search range, and (x1, y1) is the coordinate of the shifted macroblock group.

[0049] Specifically, the motion recognition model preprocesses the time-series image frame sequence in sequence to obtain a preprocessed image sequence, then grayscales the preprocessed image sequence to obtain a grayscale image sequence, calculates the grayscale value of each grayscale image according to the grayscale image sequence and the color channel weight, calculates the grayscale difference between two consecutive frames of the grayscale image according to the grayscale value, compares the grayscale difference with a preset threshold to obtain a change frame, and gathers each of the change frames to obtain the motion frame set; the preprocessing includes denoising and contrast enhancement.

[0050] Specifically, the recognition model is implemented based on a convolutional neural network and is used to determine whether a person appears in the temporal image frame sequence. The specific calculation steps are as follows:

[0051] S301: Preprocessing the time-series image frame sequence to obtain a preprocessed image sequence, wherein the preprocessing includes image grayscale conversion, denoising, and image enhancement;

[0052] S302: Extracting features according to the preprocessed image sequence to obtain feature data, wherein the feature data includes shape features and texture features;

[0053] S303: performing dimensionality reduction processing according to the feature data to obtain dimensionality reduction features, wherein the dimensionality reduction processing includes average normalization, standardization, zero component analysis, and dimensionality reduction calculation;

[0054] S304: Obtain a recognition result by calculating a classifier according to the dimension reduction feature, and label the temporal image frame sequence according to the recognition result to obtain the recognition frame set.

[0055] Specifically, the elimination model first obtains a misjudgment result according to the motion frame set through a light-sensitive diagnosis algorithm, eliminates the motion frame set according to the misjudgment result to obtain a purified frame set, and traverses and filters the purified frame set according to the timestamp of the identified frame set to obtain the target motion frame set. The calculation formula of the light-sensitive diagnosis algorithm is:

[0056]

[0057] Where x, y are the pixel coordinates, A is the image area, S(x, y, t) is the grayscale value of the pixel (x, y) at time t, a is the time difference, c is the adjustment coefficient, b is the number of pixels in the detection area, and N is the preset threshold.

[0058] In this embodiment, the photosensitive diagnosis algorithm is used to remove the change in grayscale value of the same pixel in two consecutive frames caused by the change in illumination, thereby causing movement misjudgment.

[0059] Specifically, the motion extraction model obtains a grayscale frame set by graying the target frame set, obtains a divided frame set by performing macroblock division on the grayscale frame set according to a preset macroblock width, calculates the grayscale value difference between two consecutive frames according to the divided frame set, compares the grayscale value difference with a preset threshold value to obtain a motion macroblock determination result, labels the macroblocks according to the motion macroblock determination result to obtain motion macroblocks, and aggregates adjacent motion macroblocks to obtain the motion macroblock group set.

[0060] Specifically, the character extraction model is implemented based on the recognition model. The character recognition result set is calculated by the recognition model according to the target motion frame set, the coordinate domain of the character recognition result set is obtained, the coordinate domain is compared with a preset coordinate frame threshold to obtain a comparison result, and the character recognition result set is screened according to the comparison result to obtain the target person.

[0061] Specifically, the rotation angle includes a horizontal rotation angle and a vertical rotation angle. The rotation angle model determines the movement direction of the PTZ machine by calculating the coordinate difference between the target macroblock group and the measured macroblock group, and obtains the horizontal rotation angle and the vertical rotation angle according to the coordinates of the measured macroblock group. The horizontal rotation angle calculation formula is:

[0062]

[0063] Among them, θ x is the horizontal rotation angle, x is the measured macroblock group horizontal coordinate, w is the picture width, and θx0 is the horizontal field angle;

[0064] The vertical rotation angle calculation formula is:

[0065]

[0066] Among them, θ y is the horizontal rotation angle, y is the vertical coordinate of the measured macroblock group, h is the picture height, and θy0 is the vertical field angle.

[0067] A pan-tilt machine tracking method based on motion detection and neural network and a pan-tilt machine tracking system based on motion detection and neural network, comprising: an image acquisition module, an extraction module, a comparison module, and a tracking module;

[0068] The image acquisition module is used to acquire statistical image data of motion detection through a pan-tilt machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model;

[0069] The extraction module is used to calculate a target motion frame set according to the motion frame set and the identification frame set through a elimination model, calculate a motion macroblock set according to the target motion frame set through a motion extraction model, and calculate a target person according to the target motion frame set through a person extraction model;

[0070] The comparison module is used to obtain a reference coordinate set according to the motion macroblock set, obtain a target coordinate according to the target person, and traverse and filter the reference coordinate set according to the target coordinate to obtain a target macroblock set;

[0071] The tracking module is used to obtain real-time image frames through the gimbal machine, calculate the measured macroblock group set based on the real-time image frames through the motion extraction model, calculate the shifted macroblock group based on the target macroblock group and the measured macroblock group through a matching algorithm, calculate the rotation angle based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal machine for tracking.

[0072] In this embodiment, when the intelligent terminal generates an adaptive signal according to the rotation angle to control the PTZ machine to track, it is necessary to perform motion compensation according to the coordinates of the shifted macroblock group to ensure that the tracking target is always in the center of the picture, wherein the motion compensation includes central area compensation, reverse rotation compensation, and edge compensation;

[0073] The center area compensation is used to prevent shaking and improve user experience. When a moving object approaches the center of the picture, it is not necessary to precisely adjust it to the central macroblock of the picture. Instead, the adjustment is stopped after the moving object enters the central area. This can reduce unnecessary motor rotation while ensuring picture stability, thus extending the service life of the device.

[0074] The reverse rotation compensation is that in some cases, the direction of a person's movement may be opposite to the direction in which the motor is supposed to rotate. For example, when a person moves to the left in the picture, if the motor is also rotating to the left at the same time, the person may quickly leave the picture. To solve this problem, when it is detected that the direction of a person's movement is opposite to the direction in which the motor rotates, the motor will be paused in this frame, and the motor will be adjusted according to the new movement coordinates after waiting for one frame. This ensures that the person remains visible in the picture and improves the accuracy of tracking.

[0075] The edge compensation is when the moving target moves at a fast speed and the motor cannot keep up in time. The moving target may stay at the edge of the picture and rotate in the same direction as the motor. In order to deal with this situation, an edge compensation strategy is also needed. When this scenario is detected, the algorithm will speed up the rotation speed of the motor to ensure that the moving target can remain in the central area of ​​the picture, thereby providing a smoother and more stable tracking experience.

[0076] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the pan / tilt machine tracking method based on motion detection and neural network as described above is implemented.

[0077] A storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the pan-tilt machine tracking method based on motion detection and neural network as described above when executed by a computer processor.

[0078] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0079] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0080] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0081] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A PTZ tracking method based on motion detection and neural network, characterized in that: include: S1: Obtain statistical image data of motion detection through a PTZ machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model; S2: Calculate a target motion frame set by using a elimination model according to the motion frame set and the identification frame set, calculate a motion macroblock set by using a motion extraction model according to the target motion frame set, and calculate a target person by using a person extraction model according to the target motion frame set; S3: acquiring a reference coordinate set according to the motion macroblock set, acquiring a target coordinate according to the target person, and traversing and screening the reference coordinate set according to the target coordinate to obtain a target macroblock set; S4: real-time image frames are acquired through the gimbal, a measured macroblock group set is calculated based on the real-time image frames through the motion extraction model, a shifted macroblock group is calculated based on the target macroblock group and the measured macroblock group set through a matching algorithm, a rotation angle is calculated based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal for tracking.

2. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The motion recognition model preprocesses the time-series image frame sequence in sequence to obtain a preprocessed image sequence, then grayscales the preprocessed image sequence to obtain a grayscale image sequence, calculates the grayscale value of each grayscale image based on the grayscale image sequence and the color channel weight, calculates the grayscale difference between two consecutive frames of the grayscale image based on the grayscale value, compares the grayscale difference with a preset threshold to obtain a change frame, and aggregates each of the change frames to obtain the motion frame set.

3. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The recognition model is implemented based on a convolutional neural network and is used to determine whether a person appears in the temporal image frame sequence. The specific calculation steps are as follows: S301: Preprocessing the time-series image frame sequence to obtain a preprocessed image sequence, wherein the preprocessing includes image grayscale conversion, denoising, and image enhancement; S302: extracting features from the preprocessed image sequence to obtain feature data, wherein the feature data includes shape features and texture features; S303: performing dimensionality reduction processing according to the feature data to obtain dimensionality reduction features, wherein the dimensionality reduction processing includes average normalization, standardization, zero component analysis, and dimensionality reduction calculation; S304: Obtain a recognition result by calculating a classifier according to the dimension reduction feature, and label the temporal image frame sequence according to the recognition result to obtain the recognition frame set.

4. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The elimination model first obtains misjudgment results according to the motion frame set through the light-sensitive diagnosis algorithm, eliminates the motion frame set according to the misjudgment results to obtain a purified frame set, and traverses and filters the purified frame set according to the timestamp of the identified frame set to obtain the target motion frame set. The calculation formula of the light-sensitive diagnosis algorithm is: Where x, y are the pixel coordinates, A is the image area, S(x, y, t) is the grayscale value of the pixel (x, y) at time t, a is the time difference, c is the adjustment coefficient, b is the number of pixels in the detection area, and N is the preset threshold.

5. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The motion extraction model obtains a grayscale frame set by graying the target frame set, obtains a divided frame set by dividing the grayscale frame set into macroblocks according to a preset macroblock width, calculates the grayscale value difference between two consecutive frames according to the divided frame set, compares the grayscale value difference with a preset threshold value to obtain a motion macroblock determination result, labels the macroblocks according to the motion macroblock determination result to obtain motion macroblocks, and aggregates adjacent motion macroblocks to obtain the motion macroblock group set.

6. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The character extraction model is implemented based on the recognition model. The character recognition result set is calculated by the recognition model according to the target motion frame set, the coordinate domain of the character recognition result set is obtained, the coordinate domain is compared with a preset coordinate frame threshold to obtain a comparison result, and the character recognition result set is screened according to the comparison result to obtain the target person.

7. The PTZ tracking method based on motion detection and neural network according to claim 1, characterized in that: The rotation angle includes a horizontal rotation angle and a vertical rotation angle. The rotation angle model determines the movement direction of the PTZ machine by calculating the coordinate difference between the target macroblock group and the measured macroblock group, and obtains the horizontal rotation angle and the vertical rotation angle according to the coordinates of the measured macroblock group. The calculation formula of the horizontal rotation angle is: Among them, θ x is the horizontal rotation angle, x is the measured macroblock group horizontal coordinate, w is the picture width, and θx0 is the horizontal field angle; The vertical rotation angle calculation formula is: Among them, θ y is the horizontal rotation angle, y is the vertical coordinate of the measured macroblock group, h is the picture height, and θy0 is the vertical field angle.

8. A pan-tilt machine tracking system based on motion detection and neural network using the pan-tilt machine tracking method based on motion detection and neural network as claimed in claim 1, characterized in that: include: Image acquisition module, extraction module, comparison module, tracking module; The image acquisition module is used to acquire statistical image data of motion detection through a pan-tilt machine, perform frame processing on the statistical image data to obtain an image frame sequence, add a time stamp to the image frame sequence to obtain a time-series image frame sequence, calculate a motion frame set based on the time-series image frame sequence through a motion recognition model, and calculate a recognition frame set based on the time-series image frame sequence through a recognition model; The extraction module is used to calculate a target motion frame set according to the motion frame set and the identification frame set through a elimination model, calculate a motion macroblock set according to the target motion frame set through a motion extraction model, and calculate a target person according to the target motion frame set through a person extraction model; The comparison module is used to obtain a reference coordinate set according to the motion macroblock set, obtain a target coordinate according to the target person, and traverse the reference coordinate set according to the target coordinate to obtain a target macroblock set; The tracking module is used to obtain real-time image frames through the gimbal machine, calculate the measured macroblock group set based on the real-time image frames through the motion extraction model, calculate the shifted macroblock group based on the target macroblock group and the measured macroblock group through a matching algorithm, calculate the rotation angle based on the target macroblock group and the shifted macroblock group through a rotation angle model, and the intelligent terminal generates an adaptive signal according to the rotation angle to control the gimbal machine for tracking.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the pan / tilt machine tracking method based on motion detection and neural network as described in any one of claims 1-7 is implemented.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the pan-tilt machine tracking method based on motion detection and neural network as described in any one of claims 1-7 when executed by a computer processor.

Citation Information

Patent Citations

  • Method for intelligently tracking image on screen by camera

    CN102932598A

  • Unmanned aerial vehicle intelligent identification tracking method and system

    CN111161305A

  • PTZ (Pan / Tilt / Zoom) camera active tracking method based on Dolov5 and Deepsort

    CN115988321A

  • Tracking shooting method, device and equipment of moving target and storage medium

    CN117692755A

  • Active camera video-based surveillance systems and methods

    US20050104958A1