Cradle head control method and system based on scene landmark

Abstract

The invention relates to the technical field of pan-tilt control, and particularly discloses a pan-tilt control method and system based on a scene landmark, and the method comprises the steps: obtaining the landmark distribution information of a scene, and determining the position of a pan-tilt and the parameters of the pan-tilt according to the landmark distribution information; determining the position of an edge AI analyzer and the application range of the edge AI analyzer according to the determined holder position; receiving the video acquired by the holder at regular time based on an edge AI analyzer, identifying the video, and outputting a risk type; and counting the risk type of each holder, and updating the application range of the edge AI analyzer and the video receiving period of each holder. The initial parameters of the holder are determined according to the distribution condition of the landmarks, then the edge AI analyzer is installed, the local recognition module based on the convolution kernel table and the AI recognition module based on the AI model are arranged in the edge AI analyzer, the data acquisition process of the shooting equipment on the holder is regulated and controlled according to the recognition result, and the data quality is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of pan / tilt control, and in particular to a pan / tilt control method and system based on scene landmarks. Background Art

[0002] There are two motors inside the omnidirectional gimbal, which are responsible for the up and down and left and right rotation of the gimbal respectively. It is often used to carry shooting equipment to remotely monitor the scene; most of the existing gimbals have built-in controllers that can be used to remotely control the working process of the shooting equipment.

[0003] Under the above-mentioned architecture, the existing remote monitoring process actually has a large controllable space. In the production and living areas, in fact, in most cases, the production and living areas are in a stable state. This also means that a large amount of data in the monitoring process is invalid data. How to optimize the monitoring process under the existing remote controllable architecture and improve the data quality of the data obtained in the monitoring process is the technical problem that the technical solution of the present invention wants to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a pan / tilt control method and system based on scene landmarks to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A pan / tilt control method based on scene landmarks, the method comprising:

[0007] Acquire the landmark distribution information of the scene, and determine the gimbal position and gimbal parameters according to the landmark distribution information;

[0008] Determine the location of the edge AI analyzer and its application scope according to the determined gimbal location; the application scope includes an identification of the gimbal to which the edge AI analyzer is connected;

[0009] Based on the edge AI analyzer, the video acquired by the gimbal is received periodically, the target is located in the video, the target frame is determined, the content in the target frame is identified based on a preset convolution feature table, and the risk type is output; the convolution feature table includes a preset risk type and its convolution kernel;

[0010] When the output of identifying the content in the target box based on the preset convolution feature table is empty, the content in the target box is identified based on the AI ​​recognition model in the edge AI analyzer, and the risk type is output;

[0011] Count the risk types of each gimbal, update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal;

[0012] Generate a data upload control instruction for the PTZ based on the video receiving cycle and send it to the PTZ.

[0013] As a further solution of the present invention, the step of obtaining the landmark distribution information of the scene and determining the gimbal position and gimbal parameters according to the landmark distribution information includes:

[0014] Obtain the registered landmark statistics table and query the landmark level and landmark location of each landmark;

[0015] Determine the influence range and the value of each point in the influence range according to the landmark level;

[0016] With the landmark location as the center, the influence range and the values ​​of each point in the influence range are counted to create a monitoring area with numerical values;

[0017] Superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene;

[0018] The scene is divided according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas;

[0019] Accumulate the values ​​at each point in the sub-area to obtain a numerical sum, and determine the pan / tilt definition according to the numerical sum;

[0020] The values ​​of each point in the impact range are:

[0021] Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at radius ρ; α is the preset correction coefficient.

[0022] As a further solution of the present invention, the step of determining the position of the edge AI analyzer and its application range according to the determined gimbal position includes:

[0023] Determine the resource ratio of the PTZ based on the PTZ clarity;

[0024] Clustering the PTZs according to the resource proportions;

[0025] Calculate the cluster center of each type of gimbal and select the cluster center as the location of the edge AI analyzer;

[0026] Get the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

[0027] As a further solution of the present invention, the step of periodically receiving the video acquired by the PTZ based on the edge AI analyzer, locating the target in the video, determining the target frame, identifying the content in the target frame based on a preset convolution feature table, and outputting the risk type includes:

[0028] Based on the edge AI analyzer, the video acquired by the gimbal is received regularly and converted into an image sequence;

[0029] Subtract adjacent images in sequence to obtain a difference image;

[0030] Traversing the pixel points in the difference image, marking the pixel points whose pixel values ​​are greater than a preset threshold, and connecting the adjacent marked pixel points to obtain the target area;

[0031] Eliminate the target area whose area is smaller than the preset area threshold to obtain the target frame;

[0032] The content in the target frame is identified based on the preset convolution feature table to obtain the risk type of each image.

[0033] As a further solution of the present invention, when the output of identifying the content in the target box based on the preset convolution feature table is empty, the step of identifying the content in the target box based on the AI ​​recognition model in the edge AI analyzer and outputting the risk type includes:

[0034] When the output of identifying the content in the target box based on the preset convolution feature table is empty, the image containing the target box is input into the AI ​​recognition model in the edge AI analyzer;

[0035] Read the output of the AI ​​recognition model as the risk type of the target box.

[0036] As a further solution of the present invention, the steps of counting the risk types of each gimbal, updating the application scope of the edge AI analyzer and the video receiving cycle of each gimbal include:

[0037] Risk type based on sequential reading of the PTZ;

[0038] Querying a risk value corresponding to the risk type in a preset value table, and adjusting the video receiving period of the PTZ according to the risk value;

[0039] Determine a correction rate according to the risk value and a preset risk value threshold, adjust the resource ratio of the gimbal based on the correction rate, and update the application scope of the edge AI analyzer;

[0040] The correction rate is determined as follows:

[0041] P=A(FF 0 ), where P is the correction rate, F is the risk value, and F 0 Risk value threshold, A is a preset constant.

[0042] The technical solution of the present invention also provides a pan / tilt control system based on scene landmarks, the system comprising:

[0043] A gimbal calibration module, used to obtain the landmark distribution information of the scene, and determine the gimbal position and gimbal parameters according to the landmark distribution information;

[0044] An analyzer calibration module, used to determine the position of the edge AI analyzer and its application range according to the determined gimbal position; the application range includes the identification of the gimbal to which the edge AI analyzer is connected;

[0045] A convolution recognition module is used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, locate the target in the video, determine the target frame, identify the content in the target frame based on a preset convolution feature table, and output the risk type; the convolution feature table includes a preset risk type and its convolution kernel;

[0046] An AI recognition module is used to recognize the content in the target box based on the AI ​​recognition model in the edge AI analyzer and output the risk type when the output of recognizing the content in the target box based on the preset convolution feature table is empty;

[0047] The periodic update module is used to count the risk types of each gimbal, update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal;

[0048] The real-time control module is used to generate data upload control instructions for the pan / tilt head based on the video receiving cycle and send them to the pan / tilt head.

[0049] As a further solution of the present invention, the pan / tilt calibration module includes:

[0050] A location query unit, used to obtain a statistical table of registered landmarks and query the landmark level and landmark location of each landmark;

[0051] A first calculation unit, used to determine the influence range and the value of each point in the influence range according to the landmark level;

[0052] The monitoring area determination unit is used to take the landmark position as the center, count the values ​​of the influence range and each point in the influence range, and create a monitoring area containing values;

[0053] The second calculation unit is used to superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene;

[0054] A segmentation unit, used to segment the scene according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas;

[0055] A third calculation unit is used to accumulate the values ​​at each point in the sub-area to obtain a numerical sum, and determine the pan / tilt definition according to the numerical sum;

[0056] The values ​​of each point in the impact range are:

[0057] Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at radius ρ; α is the preset correction coefficient.

[0058] As a further solution of the present invention, the analyzer calibration module includes:

[0059] A ratio determination unit, used to determine the resource ratio of the PTZ according to the PTZ definition;

[0060] A PTZ clustering unit, used for clustering PTZs according to the resource proportions;

[0061] The central application unit is used to calculate the cluster center of each type of gimbal and select the cluster center as the location of the edge AI analyzer;

[0062] The scope determination unit is used to obtain the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

[0063] As a further solution of the present invention, the convolution recognition module includes:

[0064] A video conversion unit, used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, and convert the video into an image sequence;

[0065] An image difference unit, used for sequentially difference adjacent images to obtain a difference image;

[0066] A pixel point analysis unit, used for traversing the pixel points in the difference image, marking the pixel points whose pixel values ​​are greater than a preset threshold, and connecting the adjacent marked pixel points to obtain the target area;

[0067] A target elimination unit is used to eliminate target areas whose areas are smaller than a preset area threshold to obtain a target frame;

[0068] The content recognition unit is used to recognize the content in the target frame based on a preset convolution feature table to obtain the risk type of each image.

[0069] Compared with the prior art, the beneficial effects of the present invention are: the present invention determines the initial parameters of the gimbal according to the distribution of landmarks, and then installs an edge AI analyzer, in which a local recognition module based on a convolution kernel table and an AI recognition module based on an AI model are built into the edge AI analyzer, and the data acquisition process of the shooting device on the gimbal is regulated according to the recognition results, thereby optimizing the data quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0071] Figure 1 The flowchart of the pan / tilt control method based on scene landmarks.

[0072] Figure 2 This is a flowchart of the first sub-process of the pan / tilt control method based on scene landmarks.

[0073] Figure 3 This is a block diagram of the second sub-process of the pan / tilt control method based on scene landmarks.

[0074] Figure 4 This is the third sub-process flowchart of the pan / tilt control method based on scene landmarks.

[0075] Figure 5 This is the fourth sub-process flowchart of the pan / tilt control method based on scene landmarks.

[0076] Figure 6 This is the fifth sub-process flowchart of the pan / tilt control method based on scene landmarks.

[0077] Figure 7 This is a structural block diagram of the pan-tilt control system based on scene landmarks. DETAILED DESCRIPTION

[0078] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0079] Figure 1 The flowchart of the pan-tilt control method based on scene landmarks is shown in FIG. 1 . In an embodiment of the present invention, a pan-tilt control method based on scene landmarks includes:

[0080] Step S100: Acquire the landmark distribution information of the scene, and determine the PTZ position and its PTZ parameters according to the landmark distribution information;

[0081] The pan-tilt head is an installation platform composed of two AC motors or DC motors, which can move horizontally and vertically, and on which a shooting device is installed. In this application, the pan-tilt head refers to a mounting platform for the shooting device and a collection of shooting devices. In other words, it is regarded as a shooting device that can adjust the position and angle; the scene refers to the area that needs to be monitored, including living areas and production areas.

[0082] Each area has objects that need to be monitored, called landmarks. The landmark distribution information of the scene is obtained, and the gimbal position and gimbal parameters are determined based on the landmark distribution information. The gimbal position is the location of the installation platform, and the gimbal parameters are the clarity of the shooting equipment.

[0083] Step S200: determining the position of the edge AI analyzer and its application scope according to the determined gimbal position; the application scope includes the identifier of the gimbal to which the edge AI analyzer is connected;

[0084] There is a data interaction channel between the edge AI analyzer and the gimbal. The position of the edge AI analyzer can be determined according to the determined gimbal position. The gimbal corresponding to each edge AI analyzer is the application scope of the edge AI analyzer; the application scope can be composed of the identifier of the gimbal connected to the edge AI analyzer, and the identifier is generally a number.

[0085] Step S300: Based on the edge AI analyzer, the video acquired by the PTZ is received periodically, the target is located in the video, the target frame is determined, the content in the target frame is identified based on a preset convolution feature table, and the risk type is output; the convolution feature table includes a preset risk type and its convolution kernel;

[0086] For each edge AI analyzer, there is data interaction between it and a part of the gimbal. It receives the video acquired by the gimbal at a regular interval, locates the target in each video, obtains the target frame, and first identifies the content in the target frame based on the preset convolution feature table to obtain the risk type. This process is essentially an image traversal and comparison process and does not involve the AI ​​recognition process. The convolution kernel table is a preset table. The staff pre-acquires images of different risk types, identifies the images, and then extracts image features, which are called convolution kernels.

[0087] Step S400: When the output of identifying the content in the target box based on the preset convolution feature table is empty, identifying the content in the target box based on the AI ​​recognition model in the edge AI analyzer and outputting the risk type;

[0088] The target recognition process based on the convolution feature table can be performed locally. In the existing technology, the recognition speed may not be much different, but the amount of resources consumed is less. The disadvantage of the target recognition process based on the convolution feature table is that it is impossible to identify risk types that have not been counted in advance. At this time, after introducing the AI ​​recognition model, unknown targets can be identified, which greatly improves the comprehensiveness of recognition.

[0089] Step S500: Count the risk types of each gimbal, and update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal;

[0090] After identifying the risk type, the scene risk warning process has been completed. On this basis, considering the working pressure of each edge AI analyzer, this application introduces an update process. For a certain edge AI analyzer, the risk types corresponding to each gimbal are counted. The more risk types there are, the shorter the video receiving cycle of the corresponding gimbal will be, and the more videos will be obtained. Correspondingly, the number of gimbals connected to the edge AI analyzer will decrease.

[0091] Step S600: Generate a data upload control instruction for the PTZ based on the video receiving cycle and send it to the PTZ;

[0092] After obtaining the video receiving cycle of each pan / tilt, the video receiving cycle is converted into a data upload control instruction and sent to the pan / tilt, so that the working process of the pan / tilt can be remotely controlled.

[0093] Figure 2 The first sub-process flowchart of the pan / tilt control method based on scene landmarks is as follows: the steps of obtaining the landmark distribution information of the scene and determining the pan / tilt position and pan / tilt parameters according to the landmark distribution information include:

[0094] Step S101: Obtain a registered landmark statistics table, and query the landmark level and landmark location of each landmark;

[0095] Step S102: determining the influence range and the value of each point in the influence range according to the landmark level;

[0096] Step S103: Taking the landmark position as the center, counting the values ​​of the influence range and each point in the influence range, and creating a monitoring area containing numerical values;

[0097] Step S104: superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene;

[0098] Step S105: dividing the scene according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas;

[0099] Step S106: accumulating the values ​​at each point in the sub-area to obtain a numerical sum, and determining the pan / tilt definition according to the numerical sum;

[0100] The values ​​of each point in the impact range are:

[0101] Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at radius ρ; α is the preset correction coefficient.

[0102] The value of each point can also be applied with a two-dimensional Gaussian distribution model, which has higher smoothness.

[0103] In an example of the technical solution of the present invention, landmarks are known data, which are pre-stored in a pre-registered landmark statistics table, including the landmark position and landmark level of each landmark. The landmark level represents the importance of the landmark. The influence range is determined based on the landmark level. The higher the landmark level, the larger the influence range. Combined with the landmark position, an influence area can be determined, which is called a monitoring area. The monitoring area is generally circular.

[0104] Superimpose the monitoring areas of all devices to obtain the values ​​of each point in the scene. The value at each point reflects the importance of the corresponding point. Based on this, obtain the monitoring range of the gimbal. This application requires global monitoring of the entire scene. The set of monitoring ranges of all gimbals must be larger than the scene. Therefore, the entire area of ​​the scene is divided by the monitoring range of the gimbal to obtain sub-areas.

[0105] Since a numerical value representing the importance has been calculated for each point, the values ​​at each point in the sub-area can be accumulated to obtain the numerical sum, and the pan-tilt clarity can be determined based on the numerical sum; the pan-tilt clarity is proportional to the numerical sum; that is, the more important the area corresponding to the pan-tilt is, the higher the clarity will be.

[0106] Figure 3 The second sub-flow chart of the pan-tilt control method based on scene landmarks, wherein the step of determining the position of the edge AI analyzer and its application range according to the determined pan-tilt position includes:

[0107] Step S201: determining the resource ratio of the PTZ according to the PTZ definition;

[0108] Step S202: clustering the PTZs according to the resource proportions;

[0109] Step S203: Calculate the cluster center of each type of gimbal, and select the cluster center as the location of the edge AI analyzer;

[0110] Step S204: Obtain the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

[0111] The higher the clarity of the gimbal, the higher the amount of resources required to process the corresponding video, and the higher the resource share. By clustering the gimbals according to the resource share, multiple types of gimbals can be obtained. For a type of gimbal corresponding to a higher resource share, the performance of the selected edge AI analyzer will be higher. On the contrary, for a type of gimbal corresponding to a lower resource share, the performance of the selected edge AI analyzer will be lower. In order to minimize the transmission pressure of the data transmission process (between the gimbal and the edge AI analyzer), the cluster center is selected as the position of the edge AI analyzer, and the label of the gimbal of the corresponding class of the edge AI analyzer is obtained as the application scope of the edge AI analyzer.

[0112] Figure 4 The third sub-flow diagram of the pan-tilt control method based on scene landmarks, wherein the edge AI analyzer periodically receives the video acquired by the pan-tilt, locates the target in the video, determines the target frame, identifies the content in the target frame based on a preset convolution feature table, and outputs the risk type, comprises the following steps:

[0113] Step S301: The edge AI analyzer periodically receives the video acquired by the PTZ and converts the video into an image sequence;

[0114] Step S302: subtract adjacent images in sequence to obtain a difference image;

[0115] Step S303: traverse the pixel points in the difference image, mark the pixel points whose pixel values ​​are greater than a preset threshold, and connect adjacent marked pixel points to obtain the target area;

[0116] Step S304: Eliminate the target area whose area is smaller than a preset area threshold to obtain a target frame;

[0117] Step S305: Identify the content in the target frame based on a preset convolution feature table to obtain the risk type of each image.

[0118] In an example of the technical solution of the present invention, a specific description is given of the target positioning process and the application process of the convolution feature table. The edge AI analyzer periodically receives the video acquired by the gimbal, converts the video into an image sequence (excluding the audio), and subtracts the adjacent images in turn to obtain a difference image. For the surveillance video, only the pixels corresponding to the moving object will have differences, and the other pixels are close to zero after subtraction. The pixels in the difference image are traversed, and the pixels with pixel values ​​greater than a preset threshold are marked. The adjacent marked pixels are connected to obtain the target area. The target area is the active area where the moving object is located. The target area (noise) with a particularly small area is eliminated, and the remaining target area is used as the area of ​​the real moving object. The matrix shape of the area is obtained, which is called the target frame. The content in the target frame is identified based on the preset convolution feature table. After the match is successful, the risk type corresponding to the target frame is obtained.

[0119] Figure 5 The fourth sub-flow chart of the pan / tilt control method based on scene landmarks, wherein when the output of identifying the content in the target box based on the preset convolution feature table is empty, the step of identifying the content in the target box based on the AI ​​recognition model in the edge AI analyzer and outputting the risk type includes:

[0120] Step S401: when the output of identifying the content in the target frame based on the preset convolution feature table is empty, inputting the image containing the target frame into the AI ​​recognition model in the edge AI analyzer;

[0121] Step S402: Read the output of the AI ​​recognition model as the risk type of the target box.

[0122] When the output of identifying the content in the target box based on the preset convolution feature table is empty, that is, when identifying the content in the target box based on the preset convolution feature table, the match fails. At this time, the AI ​​recognition model in the edge AI analyzer is applied to identify the target. On the basis that the AI ​​capability is already strong enough, the recognition breadth and recognition depth of the present application are greatly improved.

[0123] It is worth mentioning that in actual applications, since there are only a limited number of risk types, the convolutional feature table is sufficient to cover most types, and the application frequency of AI recognition models is actually very low.

[0124] Figure 6 The fifth sub-flow chart of the pan-tilt control method based on scene landmarks, wherein the steps of counting the risk types of each pan-tilt, updating the application scope of the edge AI analyzer and the video receiving cycle of each pan-tilt include:

[0125] Step S501: Read the risk type of the PTZ based on the image sequence;

[0126] Step S502: querying a risk value corresponding to the risk type in a preset value table, and adjusting the video receiving period of the PTZ according to the risk value;

[0127] Step S503: determining a correction rate according to the risk value and a preset risk value threshold, adjusting the resource ratio of the gimbal based on the correction rate, and updating the application scope of the edge AI analyzer;

[0128] The correction rate is determined as follows:

[0129] P=A(FF 0 ), where P is the correction rate, F is the risk value, and F 0 Risk value threshold, A is a preset constant.

[0130] In an example of the technical solution of the present invention, a negative feedback adjustment scheme is provided, which reads the risk type of the gimbal based on the image sequence, and the staff presets a numerical value representing the degree of risk for each risk type, that is, the risk value, and queries the risk value corresponding to the risk type. The video receiving period of the gimbal is adjusted according to the risk value, and the video receiving period is inversely proportional to the risk value. The larger the risk value, the shorter the video receiving period; further, the correction rate is determined according to the risk value, and the resource ratio of the gimbal is adjusted, which affects the clustering process of the gimbal, that is, which edge AI analyzers manage different gimbals. The performance of different edge AI analyzers is different. The higher the risk value, the higher the corresponding resource ratio.

[0131] Figure 7 FIG. 1 is a structural block diagram of a pan-tilt control system based on scene landmarks. In an embodiment of the present invention, a pan-tilt control system based on scene landmarks, the system 10 includes:

[0132] The gimbal calibration module 11 is used to obtain the landmark distribution information of the scene and determine the gimbal position and gimbal parameters according to the landmark distribution information;

[0133] An analyzer calibration module 12, used to determine the position of the edge AI analyzer and its application range according to the determined gimbal position; the application range includes the identification of the gimbal to which the edge AI analyzer is connected;

[0134] The convolution recognition module 13 is used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, locate the target in the video, determine the target frame, identify the content in the target frame based on a preset convolution feature table, and output the risk type; the convolution feature table includes the preset risk type and its convolution kernel;

[0135] The AI ​​recognition module 14 is used to recognize the content in the target box based on the AI ​​recognition model in the edge AI analyzer and output the risk type when the output of recognizing the content in the target box based on the preset convolution feature table is empty;

[0136] The periodic update module 15 is used to count the risk types of each gimbal, update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal;

[0137] The real-time control module 16 is used to generate a data upload control instruction of the PTZ based on the video receiving cycle and send it to the PTZ.

[0138] Furthermore, the gimbal calibration module 11 includes:

[0139] A location query unit, used to obtain a statistical table of registered landmarks and query the landmark level and landmark location of each landmark;

[0140] A first calculation unit, used to determine the influence range and the value of each point in the influence range according to the landmark level;

[0141] The monitoring area determination unit is used to take the landmark position as the center, count the values ​​of the influence range and each point in the influence range, and create a monitoring area containing values;

[0142] The second calculation unit is used to superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene;

[0143] A segmentation unit, used to segment the scene according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas;

[0144] A third calculation unit is used to accumulate the values ​​at each point in the sub-area to obtain a numerical sum, and determine the pan / tilt definition according to the numerical sum;

[0145] The values ​​of each point in the impact range are:

[0146] Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at radius ρ; α is the preset correction coefficient.

[0147] Specifically, the analyzer calibration module 12 includes:

[0148] A ratio determination unit, used to determine the resource ratio of the PTZ according to the PTZ definition;

[0149] A PTZ clustering unit, used for clustering PTZs according to the resource proportions;

[0150] The central application unit is used to calculate the cluster center of each type of gimbal and select the cluster center as the location of the edge AI analyzer;

[0151] The scope determination unit is used to obtain the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

[0152] Furthermore, the convolution recognition module 13 includes:

[0153] A video conversion unit, used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, and convert the video into an image sequence;

[0154] An image difference unit, used for sequentially difference adjacent images to obtain a difference image;

[0155] A pixel point analysis unit, used for traversing the pixel points in the difference image, marking the pixel points whose pixel values ​​are greater than a preset threshold, and connecting the adjacent marked pixel points to obtain the target area;

[0156] A target elimination unit is used to eliminate target areas whose areas are smaller than a preset area threshold to obtain a target frame;

[0157] The content recognition unit is used to recognize the content in the target frame based on a preset convolution feature table to obtain the risk type of each image.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A pan / tilt control method based on scene landmarks, characterized in that: The method comprises: Acquire the landmark distribution information of the scene, and determine the gimbal position and gimbal parameters according to the landmark distribution information; Determine the location of the edge AI analyzer and its application scope according to the determined gimbal location; the application scope includes an identification of the gimbal to which the edge AI analyzer is connected; Based on the edge AI analyzer, the video acquired by the gimbal is received periodically, the target is located in the video, the target frame is determined, the content in the target frame is identified based on a preset convolution feature table, and the risk type is output; the convolution feature table includes a preset risk type and its convolution kernel; When the output of identifying the content in the target box based on the preset convolution feature table is empty, the content in the target box is identified based on the AI ​​recognition model in the edge AI analyzer, and the risk type is output; Count the risk types of each gimbal, update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal; Generate a data upload control instruction for the PTZ based on the video receiving cycle and send it to the PTZ.

2. The pan / tilt control method based on scene landmarks according to claim 1, characterized in that: The step of acquiring the landmark distribution information of the scene and determining the pan / tilt position and pan / tilt parameters thereof according to the landmark distribution information comprises: Obtain the registered landmark statistics table and query the landmark level and landmark location of each landmark; Determine the influence range and the value of each point in the influence range according to the landmark level; With the landmark location as the center, the influence range and the values ​​of each point in the influence range are counted to create a monitoring area with numerical values; Superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene; The scene is divided according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas; Accumulate the values ​​at each point in the sub-area to obtain a numerical sum, and determine the pan / tilt definition according to the numerical sum; The values ​​of each point in the impact range are: Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at the radius ρ; α is the preset correction coefficient.

3. The pan / tilt control method based on scene landmarks according to claim 1, characterized in that: The step of determining the position of the edge AI analyzer and its application range according to the determined gimbal position includes: Determine the resource ratio of the PTZ based on the PTZ clarity; Clustering the PTZs according to the resource proportions; Calculate the cluster center of each type of gimbal and select the cluster center as the location of the edge AI analyzer; Get the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

4. The pan / tilt control method based on scene landmarks according to claim 1, characterized in that: The step of periodically receiving the video acquired by the PTZ based on the edge AI analyzer, locating the target in the video, determining the target frame, identifying the content in the target frame based on a preset convolution feature table, and outputting the risk type includes: Based on the edge AI analyzer, the video acquired by the gimbal is received regularly and converted into an image sequence; Subtract adjacent images in sequence to obtain a difference image; Traversing the pixel points in the difference image, marking the pixel points whose pixel values ​​are greater than a preset threshold, and connecting the adjacent marked pixel points to obtain the target area; Eliminate the target area whose area is smaller than the preset area threshold to obtain the target frame; The content in the target frame is identified based on the preset convolution feature table to obtain the risk type of each image.

5. The pan / tilt control method based on scene landmarks according to claim 1, characterized in that: When the output of identifying the content in the target box based on the preset convolution feature table is empty, the step of identifying the content in the target box based on the AI ​​recognition model in the edge AI analyzer and outputting the risk type includes: When the output of identifying the content in the target box based on the preset convolution feature table is empty, the image containing the target box is input into the AI ​​recognition model in the edge AI analyzer; Read the output of the AI ​​recognition model as the risk type of the target box.

6. The pan / tilt control method based on scene landmarks according to claim 1, characterized in that: The steps of counting the risk types of each gimbal, updating the application scope of the edge AI analyzer and the video receiving cycle of each gimbal include: Risk type based on sequential reading of the PTZ; Querying a risk value corresponding to the risk type in a preset value table, and adjusting the video receiving period of the PTZ according to the risk value; Determine a correction rate according to the risk value and a preset risk value threshold, adjust the resource ratio of the gimbal based on the correction rate, and update the application scope of the edge AI analyzer; The correction rate is determined as follows: P=A(F-F0); where P is the correction rate, F is the risk value, F0 is the risk value threshold, and A is a preset constant.

7. A pan / tilt control system based on scene landmarks, characterized in that: The system comprises: A gimbal calibration module, used to obtain the landmark distribution information of the scene, and determine the gimbal position and gimbal parameters according to the landmark distribution information; An analyzer calibration module, used to determine the position of the edge AI analyzer and its application range according to the determined gimbal position; the application range includes the identification of the gimbal to which the edge AI analyzer is connected; A convolution recognition module is used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, locate the target in the video, determine the target frame, identify the content in the target frame based on a preset convolution feature table, and output the risk type; the convolution feature table includes a preset risk type and its convolution kernel; An AI recognition module is used to recognize the content in the target box based on the AI ​​recognition model in the edge AI analyzer and output the risk type when the output of recognizing the content in the target box based on the preset convolution feature table is empty; The periodic update module is used to count the risk types of each gimbal, update the application scope of the edge AI analyzer and the video receiving cycle of each gimbal; The real-time control module is used to generate data upload control instructions for the pan / tilt head based on the video receiving cycle and send them to the pan / tilt head.

8. The scene landmark-based PTZ control system according to claim 7, characterized in that: The pan / tilt calibration module comprises: A location query unit, used to obtain a statistical table of registered landmarks and query the landmark level and landmark location of each landmark; A first calculation unit, used to determine the influence range and the value of each point in the influence range according to the landmark level; The monitoring area determination unit is used to take the landmark position as the center, count the values ​​of the influence range and each point in the influence range, and create a monitoring area containing values; The second calculation unit is used to superimpose the monitoring areas of all devices to obtain the values ​​at each point in the scene; A segmentation unit, used to segment the scene according to the inscribed rectangle of the monitoring range of the PTZ to obtain sub-areas; A third calculation unit is used to accumulate the values ​​at each point in the sub-area to obtain a numerical sum, and determine the pan / tilt definition according to the numerical sum; The values ​​of each point in the impact range are: Where N is the center value determined by the equipment level, R is the range radius determined by the equipment level, I(ρ) represents the value of the point at the radius ρ; α is the preset correction coefficient.

9. The pan / tilt control system based on scene landmarks according to claim 7, characterized in that: The analyzer calibration module comprises: A ratio determination unit, used to determine the resource ratio of the PTZ according to the PTZ definition; A PTZ clustering unit, used for clustering PTZs according to the resource proportions; The central application unit is used to calculate the cluster center of each type of gimbal and select the cluster center as the location of the edge AI analyzer; The scope determination unit is used to obtain the label of the gimbal of the corresponding class of the edge AI analyzer as the application scope of the edge AI analyzer.

10. The pan / tilt control system based on scene landmarks according to claim 7, characterized in that: The convolution recognition module comprises: A video conversion unit, used to periodically receive the video acquired by the PTZ based on the edge AI analyzer, and convert the video into an image sequence; An image difference unit, used for sequentially difference adjacent images to obtain a difference image; A pixel point analysis unit, used for traversing the pixel points in the difference image, marking the pixel points whose pixel values ​​are greater than a preset threshold, and connecting the adjacent marked pixel points to obtain the target area; A target elimination unit is used to eliminate target areas whose areas are smaller than a preset area threshold to obtain a target frame; The content recognition unit is used to recognize the content in the target frame based on a preset convolution feature table to obtain the risk type of each image.

Images