Image-assisted broadcasting method and device based on position information verification

Abstract

The present invention provides an image-assisted directing method and device based on position information verification. The method comprises: using a target detection model to perform target recognition on each channel of video images captured in real time by each camera device, and obtaining a target recognition result including target category, position coordinates in the image, and confidence; obtaining the position information and time information of each camera device when capturing each frame of the image, the position information including the position coordinates of the camera device, the azimuth and pitch angle of the optical axis, and the position coordinates of the target; based on the target recognition result and position information, scoring and sorting the target recognition results of each channel of video images by target, and optimizing multiple channels of video images according to the sorting. The present invention improves the accuracy of target recognition in live broadcast tasks by adding the processing steps of position information verification and fusing image recognition results, and the response speed is better than the traditional manual directing working mode.

Description

Technical Field

[0001] The present invention belongs to the technical field of image-assisted directing, and in particular relates to an image-assisted directing method and device based on position information verification. Background Art

[0002] In a live TV broadcast, multiple cameras are usually used to shoot from different angles, making the program more comprehensive and better looking. The director is responsible for selecting one channel from multiple video and audio signals to broadcast to the designated channel, and is the core part of completing the live broadcast task.

[0003] The rapid development of broadcasting and television technology, the gradual improvement of information and communication means, and the popularization of monitoring and portable video equipment have greatly enriched the information sources of live broadcast tasks, but also brought great challenges to directors; especially in the case of a large number of live images, many shooting targets, fast movement speed, and short event time, the director's work is very heavy, which can easily lead to the appearance of phenomena such as disconnected image selection and incomplete display of key frame images. There are currently two director modes: one is the mainstream manual director work mode, in which the operator conducts repeated and detailed deductions and exercises according to the pre-specified director plan and plan, continuously optimizes the control steps, runs in the collaborative process, and solves various problems to present a smoother director effect. This mode has the disadvantages of being time-consuming, labor-intensive, and costly. In addition, the directing effect is not ideal for sudden live events that have no plans, no simulations, and uncontrollable content. The second is the auxiliary directing working mode based on image recognition technology. It uses computer vision technology to establish a training set of target images in advance, and generates a trained classifier model by selecting a suitable machine learning training algorithm to identify targets in video images. Based on the recognition results, the optimal sorting and directing of multi-channel videos are completed. This mode is limited by the versatility, timeliness, accuracy, and hardware computing power requirements of current image recognition technology. It is unable to accurately identify targets captured by multiple cameras in TV live broadcast tasks. For example, in situations where the target in the image is partially blocked by obstacles, there are many targets to be identified and the appearance of multiple targets is consistent, and new targets that have not been trained for identification appear in the image, the target recognition accuracy is not high, and the auxiliary directing effect is not good. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an image-assisted directing method and device based on position information verification.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides an image-assisted directing method based on position information verification, comprising the following steps:

[0007] Use the target detection model to perform target recognition on each video image captured by each camera in real time, and obtain target recognition results including target category, position coordinates in the image and confidence level;

[0008] Acquire the position information and time information of each camera device when shooting each frame of image, wherein the position information includes the position coordinates of the camera device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target;

[0009] Based on the target recognition result and the position information, the target recognition result of each channel of video images is scored and sorted according to the target, and the multiple channels of video images are optimized according to the sorting.

[0010] Furthermore, the method further includes a time information correction step, which is as follows:

[0011] The difference between the time information and Beijing time is calculated, and if the difference exceeds a set threshold, the time information is corrected to Beijing time.

[0012] Furthermore, the method further comprises a data alignment step, wherein the data comprises video image data and position information data, and the method is as follows:

[0013] Get the time information of the first frame of data of each channel within the set time;

[0014] Calculate the difference between any two time information;

[0015] Calculate the maximum time interval Δt that can divide all the differences;

[0016] The frame rate of each data channel is increased to 1 / Δt through linear interpolation.

[0017] Furthermore, the method further comprises a step of clearing invalid data, wherein the data comprises video image data and location information data, and the method is as follows:

[0018] The time interval between two adjacent frames of data in each data channel is calculated. If the time interval exceeds a set threshold, the stored data in this channel is cleared.

[0019] Furthermore, different target categories identified from video images shot by the recording device at different viewing angles and distances relative to the target and under different occlusions and lighting environments constitute a derived target category set of the target; each target category is arranged in descending order according to the degree of association with the target to obtain a ranking D for each target category.

[0020] Furthermore, the method of scoring the target recognition result of each video image according to the target includes:

[0021] Scoring the target recognition result of each video image based on the position information, the closer the target is to the optical axis of the video recording device, the higher the score S1; establishing a set I1 consisting of the video recording devices corresponding to each score S1;

[0022] According to the confidence R of the target category and the ranking D of the target category in the derived target category, the target recognition result of each video image is scored. The larger R is and the smaller D is, the higher the score S2 is. A set I2 consisting of the video recording devices corresponding to each score S2 is established.

[0023] The intersection I of the sets I1 and I2 is calculated, and the target recognition results of the video images captured by each video recording device in the non-empty set I are comprehensively scored based on S1 and S2 to obtain a score S of the target recognition result.

[0024] Furthermore, the calculation method of S1 includes:

[0025] Calculate the coordinates (E, N, U) of the target in the measurement coordinate system with the recording device as the origin;

[0026] Calculate the azimuth angle and the elevation angle of the line connecting the target and the video recording device based on (E, N, U), and respectively calculate the differences Δα and Δβ between the azimuth angle and the elevation angle and the azimuth angle and the elevation angle of the optical axis of the video recording device, wherein the absolute values ​​of Δα and Δβ are both less than 180°;

[0027] Calculate S1 as follows:

[0028] S1=1-max(|x|,|y|) (1)

[0029] x=Δα / L1 (2)

[0030] y=Δβ / L2 (3)

[0031] Wherein, L1 is half of the azimuth viewing angle of the video recording device, and L2 is half of the elevation viewing angle of the video recording device.

[0032] Furthermore, the calculation method of S2 includes:

[0033] Traverse all target category information in the target recognition results of each video image, and determine in turn whether each target category belongs to a derived target category of a certain target. If not, S2=0; if so, obtain the ranking D of the target category in the derived target category to which it belongs, and calculate S2 as follows:

[0034] S2=(MD)×R (4)

[0035] Where M is the number of elements in the derived target category set, D = 0, 1, 2, ..., (M-1).

[0036] Furthermore, the calculation method of S includes:

[0037] Calculate the position coordinates (x1, y1) of the target in the image, and calculate the distance d between point (x1, y1) and point (x, y) as follows:

[0038]

[0039] Calculate S as follows:

[0040]

[0041] Where k1, k2, and k3 are weight coefficients.

[0042] In a second aspect, the present invention provides an image-assisted broadcasting device based on position information verification, comprising:

[0043] The target recognition module is used to use the target detection model to perform target recognition on each video image captured by each camera in real time, and obtain a target recognition result including the target category, position coordinates in the image and confidence level;

[0044] A position acquisition module, used to acquire the position information and time information of each camera device when shooting each frame of image, wherein the position information includes the position coordinates of the camera device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target;

[0045] The scoring and sorting module is used to score and sort the target recognition results of each video image according to the target based on the target recognition results and position information, and optimize the multiple video images according to the sorting.

[0046] Compared with the prior art, the present invention has the following beneficial effects.

[0047] The present invention uses a target detection model to perform target recognition on each video image captured in real time by each camera device, obtains a target recognition result including the target category, position coordinates in the image and confidence, obtains the position information and time information of each camera device when capturing each frame of the image, the position information includes the position coordinates of the camera device, the azimuth and pitch angle of the optical axis, and the position coordinates of the target, and scores and sorts the target recognition results of each video image according to the target based on the target recognition result and position information, and optimizes multiple video images according to the sorting, thereby realizing image-assisted directing based on position information verification. The present invention improves the accuracy of target recognition in live broadcast tasks by adding the processing steps of position information verification and fusing image recognition results, and the response speed is better than the traditional manual directing working mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention is a flowchart of an image-assisted directing method based on position information verification according to an embodiment of the present invention.

[0049] Figure 2 A schematic diagram of the hardware architecture involved in an embodiment of the present invention.

[0050] Figure 3 The present invention is a block diagram of an image-assisted broadcasting device based on position information verification according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0052] Figure 1 The present invention is a flowchart of an image-assisted directing method based on position information verification according to an embodiment of the present invention, comprising the following steps:

[0053] Step 101, using a target detection model to perform target recognition on each video image captured in real time by each video recording device, and obtaining a target recognition result including target category, position coordinates in the image and confidence level;

[0054] Step 102, obtaining the position information and time information of each camera device when shooting each frame of image, wherein the position information includes the position coordinates of the camera device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target;

[0055] Step 103, based on the target recognition result and the position information, the target recognition result of each channel of video images is scored and sorted by target, and multiple channels of video images are optimized according to the sorting.

[0056] This embodiment provides an image-assisted directing method based on position information verification. The hardware architecture diagram of this embodiment is as follows: Figure 2As shown. The video recording device in the figure has the function of real-time reading of status data such as azimuth, pitch angle, focal length, etc., and provides indicator information such as the horizontal viewing angle and vertical viewing angle of the video recording device. The base of the video recording device obtains geocentric coordinate information through geodetic mapping. The target (by setting a positioning device such as GPS or Beidou) has a real-time feedback function of positioning coordinates, providing information such as the type, characteristics, and physical size of the target. The image encoding and decoding device has image encoding and decoding functions, and completes the long-distance transmission of signals through wired / wireless networks. The image recognition server has the target recognition processing function of the captured image, and outputs the category, image coordinates, confidence, and other information of the recognized target. The preferred sorting server (software for executing the method described in this embodiment) has the function of sorting the multi-channel recorded images corresponding to the recording effect of a certain target, and has a multi-channel image acquisition and information annotation function. It can annotate the recognition results of the image recognition server and the position information of the target in the frame obtained through the coordinates and status information, and can replace the operator, matrix or selection and cutting station to continuously output the selected recorded images, and can output the sorted images separated by multiple screens to assist the director operator in controlling the equipment. The method described in this embodiment includes steps 101 to 103.

[0057] In this embodiment, step 101 is mainly used to obtain target recognition results. In this embodiment, each video image captured by each video recording device in real time is input into a trained target detection model, and the model outputs a target recognition result. The target recognition result includes the target category, the position coordinates of the target (detection frame) in the image frame, and the confidence. The target detection model can select an open source model for image target recognition with good real-time performance (such as the YOLO series model). The target categories of this embodiment include airplanes, rotorcraft, cars, ships, flames, tail smoke, light groups, spots, parachutes, people, etc.

[0058] In this embodiment, step 102 is mainly used to obtain position information. The position information includes the position coordinates of each camera device when shooting each frame of image, the azimuth and pitch angle of the optical axis, and the position coordinates of the target. The position coordinates of the camera device can adopt the coordinates of the geocentric space rectangular coordinate system (X, Y, Z) or the earth rectangular coordinate system (L, B, H), etc. The position coordinates of the target are obtained by the GPS or Beidou receiver set on the target, using the earth rectangular coordinate system coordinates (L, B, H). In order to align the data in time, it is also necessary to obtain the time information when the camera device shoots each frame of image, as well as the time information corresponding to the target position coordinate data.

[0059] In this embodiment, step 103 is mainly used to score and sort the target recognition results. Different from the prior art, the target recognition result scoring of this embodiment takes into account the position information of the recording device and the target, which can improve the accuracy of the target recognition result scoring. The scoring principle is that the closer the target is to the optical axis of the recording device, the higher the score; when the target is not within the visual range of the recording device, the target category identified from the image is not the target to be tracked, that is, the wrong target is followed. The sorting of the recognition result scores is generally carried out in order from high to low, and the higher the score, the higher the ranking. This embodiment is sorted by target, that is, the set of sorted recording devices for each target (one recording device corresponds to the target recognition result of one video image), expressed as: target 1: [device 11, device 12, ...], target 2: [device 21, device 22, ...], and so on. The sorting can replace the operator and the matrix type device or the selection and switching station to continuously output the image at the front, and can also output the sorted image separated by multiple screens. By using the above-described sorting to screen and optimize multiple video images, video images that do not capture a designated target can be eliminated, thereby reducing the director's workload and improving the director's response speed.

[0060] As an optional embodiment, the method further includes a time information correction step, the method is as follows:

[0061] The difference between the time information and Beijing time is calculated, and if the difference exceeds a set threshold range, the time information is corrected to Beijing time.

[0062] This embodiment provides a technical solution for correcting time information. This embodiment, including the following two embodiments, belongs to the data preprocessing step. This embodiment sets a threshold range of the error between the time information and Beijing time, calculates the difference between the time information in the newly received target and recording device data and Beijing time, and corrects the time information whose difference exceeds the threshold range to Beijing time. Correcting the time information can reduce the calculation error caused by the inconsistency of the clocks of various devices.

[0063] As an optional embodiment, the method further includes a data alignment step, the data includes video image data and position information data, and the method is as follows:

[0064] Get the time information of the first frame of data of each channel within the set time;

[0065] Calculate the difference between any two time information;

[0066] Calculate the maximum time interval Δt that can divide all the differences;

[0067] The frame rate of each data channel is increased to 1 / Δt through linear interpolation.

[0068] This embodiment provides a technical solution for aligning the time and frame rate of data. First, the time information of the first frame of each data channel within a set time (such as 1 second) is obtained; then the difference between any two time information is calculated; next, the maximum time interval Δt that can divide all the differences is calculated, that is, the greatest common divisor of all the differences; finally, the frame rate of each data channel is increased to 1 / Δt through linear interpolation. After the above alignment operation, the interval between two adjacent frames of data in each data channel is reduced to Δt. Therefore, the synchronization error between any two data channels is less than Δt, thereby eliminating the impact caused by the misalignment of the time and frame rate of the data.

[0069] As an optional embodiment, the method further includes a step of clearing invalid data, wherein the data includes video image data and location information data, and the method is as follows:

[0070] The time interval between two adjacent frames of data in each data channel is calculated. If the time interval exceeds a set threshold, the stored data in this channel is cleared.

[0071] This embodiment provides a technical solution for clearing invalid data. This embodiment determines whether data interruption occurs by setting a maximum interval threshold between two adjacent frames of data. If the data interval of a certain data path exceeds the set threshold, it is considered that a communication failure has occurred, and the stored data is cleared to avoid calculation errors caused by poor communication links.

[0072] In addition to the data preprocessing methods given in the above three embodiments, the data cache capacity can also be set: according to the delay time of receiving the data and the time consumed by the program calculation, the maximum number of data storage frames is set. If exceeded, the first n frames of data of this data path are cleared (n is the number of frames of the excess data) to reduce unnecessary performance consumption of the program.

[0073] As an optional embodiment, different target categories identified from video images shot by a recording device at different viewing angles and distances relative to the target and under different occlusions and lighting conditions constitute a set of derived target categories of the target; each target category is arranged in descending order according to the degree of association with the target to obtain a ranking D for each target category.

[0074] This embodiment provides a technical solution for calculating the target category ranking D. There are many derived target categories belonging to the same physical target in the target categories given above. Since the shapes and sizes of the target images captured by the recording equipment from different perspectives, different distances relative to the target, and under different occlusion and light environments are very different, the target categories identified from the image are also different. For example, the aircraft is blocked and only the wings are exposed. In the image taken from the back in the sky, only the tail flame or tail smoke may be seen; in the night image, it may be an indicator light; when the distance is too far, it may be just a point in the image. These different target categories obtained from the same physical target constitute a derived target category. Determine the degree of association between each derived target category and the real target category, and sort each derived target category in order from large to small according to the degree of association, and obtain the ranking D of the target category, D = 0, 1, 2, 3, ..., respectively representing the ranking as first, second, third, etc. For example, the ranking of the derived target categories of the aircraft is aircraft (D = 0), flame (D = 1), light group (D = 2), tail smoke (D = 3), spot (D = 4).

[0075] As an optional embodiment, the method for scoring the target recognition result of each video image according to the target includes:

[0076] Scoring the target recognition result of each video image based on the position information, the closer the target is to the optical axis of the video recording device, the higher the score S1; establishing a set I1 consisting of the video recording devices corresponding to each score S1;

[0077] According to the confidence R of the target category and the ranking D of the target category in the derived target category, the target recognition result of each video image is scored. The larger R is and the smaller D is, the higher the score S2 is. A set I2 consisting of the video recording devices corresponding to each score S2 is established.

[0078] The intersection I of the sets I1 and I2 is calculated, and the target recognition results of the video images captured by each video recording device in the non-empty set I are comprehensively scored based on S1 and S2 to obtain a score S of the target recognition result.

[0079] This embodiment provides a technical solution for scoring the target recognition results. The scoring of the target recognition results in this embodiment is related to the relative position of the recording device and the target (i.e., the position information is taken into account), and is also related to the ranking D of the target category in the derived target category to which it belongs and the confidence R of the target category. This embodiment scores based on these two aspects respectively, and the two parts of the scoring are recorded as S1 and S2 respectively, wherein the scoring principle of S1 is: the closer the target is to the optical axis of the recording device, the higher S1; the scoring principle of S2 is: the larger R is, the smaller D is, the higher S2 is. Then solve the intersection of the recording devices corresponding to these two parts, that is, the recording devices included in both parts, and perform a comprehensive scoring based on S1 and S2 corresponding to these recording devices to obtain the score S for the target recognition result.

[0080] As an optional embodiment, the calculation method of S1 includes:

[0081] Calculate the coordinates (E, N, U) of the target in the measurement coordinate system with the recording device as the origin;

[0082] Calculate the azimuth angle and the elevation angle of the line connecting the target and the video recording device based on (E, N, U), and respectively calculate the differences Δα and Δβ between the azimuth angle and the elevation angle and the azimuth angle and the elevation angle of the optical axis of the video recording device, wherein the absolute values ​​of Δα and Δβ are both less than 180°;

[0083] Calculate S1 as follows:

[0084] S1=1-max(|x|,|y|) (1)

[0085] x=Δα / L1 (2)

[0086] y=Δβ / L2 (3)

[0087] Wherein, L1 is half of the azimuth viewing angle of the video recording device, and L2 is half of the elevation viewing angle of the video recording device.

[0088] This embodiment provides a technical solution for calculating S1.

[0089] First, calculate the coordinates (E, N, U) of the target in the measurement coordinate system with the recording device as the origin. The formula is as follows:

[0090] E=-(X-X0)×sinL+(Y-Y0)×cosL

[0091] N=-(X-X0)×sinB×cosL-(Y-Y0)×sinB×sinL+(Z-Z0)×cosB

[0092] U=(X-X0)×cosB×cosL+(Y-Y0)×cosB×sinL+(Z-Z0)×sinB

[0093] Wherein, (X, Y, Z) and (X0, Y0, Z0) are the coordinates of the center point of the target and the recording device in the geocentric space rectangular coordinate system, and (L, B, H) are the coordinates of the center point of the recording device in the geocentric space rectangular coordinate system.

[0094] Secondly, the azimuth angle α and pitch angle β of the target center point are calculated based on (E, N, U). The formula is as follows:

[0095]

[0096]

[0097] Then, the differences between the azimuth angle α, the pitch angle β and the azimuth angle α0, the pitch angle β0 of the optical axis of the recording device are calculated: Δα=α-α0, Δβ=β-β0, and the absolute values ​​of Δα and Δβ are both less than 180°.

[0098] Finally, S1 is calculated according to equations (1) to (3). According to equations (1) to (3), when the absolute values ​​of x and y are both less than or equal to 1, the target center point appears in the image frame output by the recording device, and the score is greater than or equal to 0. When the absolute value of x or y is greater than 1, the score is negative. As the absolute value of x or y increases, the score decreases further (more negative), and the target center point gradually moves away from the image frame, but the target has not yet completely moved out of the image frame. Until the score decreases to -max(2×arctan(w / 2r) / L1,2×arctan(h / 2r) / L2), the target is completely moved out of the image frame, where r is the distance between the target and the center point of the device at the same time, and w and h are the maximum dimensions of the target's current posture in the horizontal and vertical directions.

[0099] As an optional embodiment, the calculation method of S2 includes:

[0100] Traverse all target category information in the target recognition results of each video image, and determine in turn whether each target category belongs to a derived target category of a certain target. If not, S2=0; if so, obtain the ranking D of the target category in the derived target category to which it belongs, and calculate S2 as follows:

[0101] S2=(MD)×R (4)

[0102] Where M is the number of elements in the derived target category set, D = 0, 1, 2, ..., (M-1).

[0103] This embodiment provides a technical solution for calculating S2. The calculation formula of S2 is as shown in formula (4). The size of S2 is proportional to the confidence and negatively correlated with the ranking D. The larger D is, the smaller S2 is. When D = 0, the factor (MD) is the largest, which is M; when D = M-1, the factor (MD) is the smallest, which is 1.

[0104] As an optional embodiment, a method for calculating S includes:

[0105] Calculate the position coordinates (x1, y1) of the target in the image, and calculate the distance d between point (x1, y1) and point (x, y) as follows:

[0106]

[0107] Calculate S as follows:

[0108]

[0109] Where k1, k2, and k3 are weight coefficients.

[0110] This embodiment is a technical solution for calculating S. This embodiment first calculates the position coordinates (x1, y1) of the target in the image; then calculates the distance d between the point (x1, y1) and the point (x, y), as shown in formula (5); and finally calculates S according to formula (6). S is equal to the weighted sum of the three parts.

[0111] Figure 3 The present invention is a schematic diagram of a composition of an image-assisted broadcasting device based on position information verification according to an embodiment of the present invention, wherein the device comprises:

[0112] The target recognition module 11 is used to use the target detection model to perform target recognition on each video image captured by each video recording device in real time, and obtain a target recognition result including the target category, position coordinates in the image and confidence level;

[0113] A position acquisition module 12 is used to acquire the position information and time information of each camera device when shooting each frame of image, wherein the position information includes the position coordinates of the camera device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target;

[0114] The scoring and sorting module 13 is used to score and sort the target recognition results of each video image according to the target based on the target recognition results and position information, and optimize the multiple video images according to the sorting.

[0115] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0116] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An image-assisted directing method based on position information verification, characterized in that: The following steps are involved: Step 101: Use a target detection model to perform target recognition on each video image captured by each video camera in real time, and obtain a target recognition result including a target category, a position coordinate in the image, and a confidence level; the target category includes different target categories recognized from video images captured by the video camera at different viewing angles and distances relative to the target and under different occlusions and light environments, and the different target categories constitute a derived target category set of the target; and arrange each target category in descending order according to the degree of association between the target category and the target, and obtain a ranking D for each target category; Step 102: obtaining the position information and time information of each video recording device when shooting each frame of the image, wherein the position information includes the position coordinates of the video recording device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target; Step 103: based on the target recognition result and the position information, the target recognition result of each channel of video images is scored and sorted by target, and multiple channels of video images are optimized according to the sorting; Methods for scoring target recognition results of each video image by target include: Step 1031: Score the target recognition result of each video image based on the position information, with the score being S1. The closer the target is to the optical axis of the video recording device, the higher the score S1. Establish a set I1 consisting of the video recording devices corresponding to each score S1. The calculation method of S1 includes: Calculate the coordinates (E, N, U) of the target in the measurement coordinate system with the recording device as the origin; Calculate the azimuth angle and the elevation angle of the line connecting the target and the video recording device based on (E, N, U), and respectively calculate the differences Δα and Δβ between the azimuth angle and the elevation angle and the azimuth angle and the elevation angle of the optical axis of the video recording device, wherein the absolute values ​​of Δα and Δβ are both less than 180°; Calculate S1 as follows: S1=1-max(|x|,|y|) (1) x=Δα / L1 (2) y=Δβ / L2 (3) Where, L1 is half of the azimuth viewing angle of the video recording device, and L2 is half of the elevation viewing angle of the video recording device; Step 1032: Score the target recognition result of each video image according to the confidence R of the target category and the ranking D of the target category in the derived target category to which it belongs, and the score is S2. The larger R is and the smaller D is, the higher the score S2 is. Establish a set I2 consisting of the video recording devices corresponding to each score S2. The calculation method of S2 includes: Traverse all target category information in the target recognition results of each video image, and determine in turn whether each target category belongs to a derived target category of a certain target. If not, S2=0; if so, obtain the ranking D of the target category in the derived target category to which it belongs, and calculate S2 as follows: S2=(MD)×R (4) Where M is the number of elements in the derived target category set, D = 0, 1, 2, ..., (M-1); Step 1033, calculating the intersection I of the sets I1 and I2, and comprehensively scoring the target recognition results of the video images captured by each video recording device in the non-empty set I based on S1 and S2 to obtain a score S of the target recognition result; The calculation methods of S include: Calculate the position coordinates (x1, y1) of the target in the image, and calculate the distance d between the point (x1, y1) and the point (x, y) as follows: Calculate S as follows: Where k1, k2, and k3 are weight coefficients.

2. The image-assisted broadcasting method based on position information verification according to claim 1, characterized in that: The method further comprises a time information correction step, which is as follows: The difference between the time information and Beijing time is calculated, and if the difference exceeds a set threshold, the time information is corrected to Beijing time.

3. The image-assisted directing method based on position information verification according to claim 1, characterized in that: The method further comprises a data alignment step, wherein the data comprises video image data and position information data, and the method is as follows: Get the time information of the first frame of data of each channel within the set time; Calculate the difference between any two time information; Calculate the maximum time interval Δt that can divide all the differences; The frame rate of each data channel is increased to 1 / Δt through linear interpolation.

4. The image-assisted directing method based on position information verification according to claim 1, characterized in that: The calculation method of S includes: Calculate the position coordinates (x1, y1) of the target in the image, and calculate the distance d between the point (x1, y1) and the point (x, y) as follows: Calculate S as follows: Where k1, k2, and k3 are weight coefficients.

5. An image-assisted broadcasting device based on position information verification, characterized in that: include: A target recognition module is used to use a target detection model to perform target recognition on each video image captured by each camera in real time, and obtain a target recognition result including a target category, a position coordinate in the image, and a confidence level; the target category includes different target categories recognized from video images captured by the camera at different viewing angles and distances relative to the target and under different occlusions and light environments, and the different target categories constitute a derived target category set of the target; and each target category is arranged in descending order according to the degree of association with the target, and a ranking D of each target category is obtained; A position acquisition module, used to acquire the position information and time information of each camera device when shooting each frame of image, wherein the position information includes the position coordinates of the camera device, the azimuth angle and pitch angle of the optical axis, and the position coordinates of the target; A scoring and sorting module, for scoring and sorting the target recognition results of each channel of video images according to the target based on the target recognition results and the position information, and optimizing the multiple channels of video images according to the sorting; Methods for scoring target recognition results of each video image by target include: Based on the position information, the target recognition result of each video image is scored, and the score is S1. The closer the target is to the optical axis of the video recording device, the higher the score S1 is; and a set I1 consisting of the video recording devices corresponding to each score S1 is established; The calculation method of S1 includes: Calculate the coordinates (E, N, U) of the target in the measurement coordinate system with the recording device as the origin; Calculate the azimuth angle and the elevation angle of the line connecting the target and the video recording device based on (E, N, U), and respectively calculate the differences Δα and Δβ between the azimuth angle and the elevation angle and the azimuth angle and the elevation angle of the optical axis of the video recording device, wherein the absolute values ​​of Δα and Δβ are both less than 180°; Calculate S1 as follows: S1=1-max(|x|,|y|) (1) x=Δα / L1 (2) y=Δβ / L2 (3) Where, L1 is half of the azimuth viewing angle of the video recording device, and L2 is half of the elevation viewing angle of the video recording device; According to the confidence R of the target category and the ranking D of the target category in the derived target category, the target recognition result of each video image is scored, and the score is S2. The larger R is and the smaller D is, the higher the score S2 is. A set I2 consisting of the video recording devices corresponding to each score S2 is established; The calculation method of S2 includes: Traverse all target category information in the target recognition results of each video image, and determine in turn whether each target category belongs to a derived target category of a certain target. If not, S2=0; if so, obtain the ranking D of the target category in the derived target category to which it belongs, and calculate S2 as follows: S2=(MD)×R (4) Where M is the number of elements in the derived target category set, D = 0, 1, 2, ..., (M-1); Calculate the intersection I of the sets I1 and I2, and perform a comprehensive score on the target recognition results of the video images captured by each video recording device in the non-empty set I based on S1 and S2 to obtain a score S of the target recognition result; The calculation method of S includes: Calculate the position coordinates (x1, y1) of the target in the image, and calculate the distance d between the point (x1, y1) and the point (x, y) as follows: Calculate S as follows: Where k1, k2, and k3 are weight coefficients.

Images