Gun and ball linkage system control method, system, equipment, medium and program product

Abstract

The invention is suitable for the technical field of monitoring equipment control, and provides a gun-dome camera linkage system control method, system and equipment, a medium and a program product, and the method comprises the steps: carrying out the coordinate correlation calibration of image pixels of a gun-type camera and a dome camera in a gun-dome camera linkage system, and detecting the pitch angle of the gun-type camera in a working process; under the condition that it is detected that the pitch angle of the gun-type camera is changed, according to the pitch angle change condition of the gun-type camera, the calibrated initial coordinate mapping relation between the gun-type camera and the dome camera is corrected, and the corrected gun-dome coordinate mapping relation between the gun-type camera and the dome camera is determined; and according to the corrected gun-dome coordinate mapping relation, controlling the dome camera to perform linkage tracking on the monitoring picture of the gun camera. In the embodiment of the invention, under the condition that the pitch angle of the gun camera changes, the coordinate mapping relation between the gun camera and the dome camera can be corrected according to the change condition of the pitch angle of the gun camera, so that the condition that the dome camera is inaccurate to track due to the change of the lens angle of the gun camera is reduced, and the accuracy of picture tracking is improved.

Description

Technical Field

[0001] The present application relates to the field of monitoring equipment control technology, and in particular to a gun-ball linkage system control method, system, equipment, medium and program product. Background Art

[0002] The gun-ball linkage system refers to a monitoring system composed of a gun-type camera device (also known as a gun camera) and a spherical camera device (a dome camera). The gun-ball linkage system uses a gun camera with a wide field of view to form a global image of the monitored area. Using the global image of the gun camera as a reference, the dome camera with a small field of view is controlled through linkage to track and output local detail images. The gun-ball linkage in the gun-ball linkage system usually relies on the gun-ball coordinate calibration, that is, the gun-ball linkage relies on the coordinate mapping relationship between the gun camera and the dome camera obtained by calibration.

[0003] The inventor discovered that after the gun-ball linkage system completes the calibration of the gun-ball coordinates, if the pitch angle of the gun camera lens changes, the coordinate mapping relationship between the gun camera and the ball camera will change, resulting in inaccurate tracking of the ball camera, resulting in low accuracy of image tracking. Summary of the invention

[0004] The present application provides a gun-ball linkage system control method, system, device, medium and program product to solve the problem in the related art that the change in the gun lens angle causes the change in the gun-ball coordinate mapping relationship, resulting in low accuracy of image tracking.

[0005] In a first aspect, an embodiment of the present application provides a gun-ball linkage system control method, comprising:

[0006] After the coordinate correlation calibration of the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, the pitch angle of the gun camera during operation is detected;

[0007] When it is detected that the elevation angle of the gun camera changes, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the elevation angle change of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined;

[0008] According to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera.

[0009] Optionally, according to the change of the pitch angle of the gun camera, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected to determine the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera, including:

[0010] According to the change of the elevation angle after the bolt is rotated, the displacement proportional coefficient of the bolt is determined, and the displacement proportional coefficient is the ratio of the change value of the elevation angle after the bolt is rotated to the maximum change value of the elevation angle of the bolt;

[0011] According to the displacement proportional coefficient, the coordinate correction data of the ball camera after the bolt rotates is determined;

[0012] The initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain a corrected gun-ball coordinate mapping relationship.

[0013] Optionally, according to the change of the elevation angle after the bolt is rotated, the displacement proportional coefficient of the bolt is determined, including:

[0014] Determine the difference between the pitch angle after the bolt is rotated and the calibrated pitch angle to obtain the pitch angle change value, the calibrated pitch angle is the pitch angle of the bolt when the coordinate association calibration is performed;

[0015] Determine the maximum change value of the elevation angle of the bolt according to the elevation angle range of the bolt;

[0016] The ratio of the pitch angle change value to the maximum pitch angle change value is determined as the displacement proportional coefficient of the bolt.

[0017] Optionally, according to the displacement proportionality coefficient, the coordinate correction data of the ball camera after the bolt rotates is determined, including:

[0018] Determine the calibration point adjustment data of the pre-calibrated ball camera, the calibration point adjustment data including the coordinate displacement values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera rotates according to the maximum change value of the pitch angle;

[0019] The displacement proportional coefficient is multiplied by the calibration point adjustment data to determine the coordinate correction data of the ball camera after the gun rotates. The coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun rotates.

[0020] Optionally, the coordinate correction data includes coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun is rotated. The initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain a corrected gun-ball coordinate mapping relationship, including:

[0021] Determine the initial coordinates of multiple calibration points of the ball camera in the initial coordinate mapping relationship;

[0022] The initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship are subtracted from the coordinate correction values ​​required for the associated adjustment of different calibration points in the coordinate correction data to obtain the corrected gun-ball coordinate mapping relationship.

[0023] Optionally, according to the change of the pitch angle of the gun camera, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected, including:

[0024] When it is determined that the pixel range of the monitoring screen after the bolt is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the bolt, and the pixel range of the calibration screen is the pixel range of the monitoring screen of the bolt during the coordinate association calibration;

[0025] When it is determined that the pixel range of the monitoring screen after the gun is rotated is inconsistent with the pixel range of the calibration screen, the pixel range of the monitoring screen after the gun is rotated is adjusted, and when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the gun's pitch angle.

[0026] In a second aspect, the embodiment of the present application provides a gun-ball linkage system, including a control device, a gun camera and a ball camera, wherein the control device is used to:

[0027] After the coordinate correlation calibration of the screen pixels of the gun camera and the ball camera, the pitch angle of the gun camera during operation is detected;

[0028] When it is detected that the elevation angle of the gun camera changes, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the elevation angle change of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined;

[0029] According to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned gun-ball linkage system control method when executing the computer program.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned gun-ball linkage system control method are implemented.

[0032] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that the computer program product includes a computer program, which, when executed by a processor, implements the steps of the above-mentioned gun-ball linkage system control method.

[0033] In one solution provided by the gun-ball linkage system control method, system, device, medium and program product, after the coordinate correlation calibration of the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, the pitch angle of the gun camera during operation is detected; when it is detected that the pitch angle of the gun camera has changed, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the change of the pitch angle of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined; according to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera. In this embodiment, when the pitch angle of the gun camera changes, the coordinate mapping relationship between the gun camera and the ball camera can be corrected according to the change of the pitch angle of the gun camera, thereby reducing the situation where the ball camera is not accurately tracked due to the change of the gun camera lens angle, and improving the accuracy of screen tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 It is a structural schematic diagram of a gun-ball linkage system in one embodiment of the present application;

[0036] Figure 2 It is a flow chart of a gun-ball linkage system control method in one embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of the pitch angle change of the bolt in one embodiment of the present application;

[0038] Figure 4 This is a schematic diagram of the monitoring screen changes of the gun machine in one embodiment of the present application;

[0039] Figure 5 yes Figure 2 A schematic diagram of an implementation process of step S20;

[0040] Figure 6 yes Figure 2 Another implementation flow diagram of step S20;

[0041] Figure 7 It is a schematic diagram of the change in coordinate position of the calibration point before and after the pitch angle of the bolt is rotated in one embodiment of the present application;

[0042] Figure 8 yes Figure 1 A structural schematic diagram of the control device;

[0043] Fig. 9 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0046] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0047] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0048] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0049] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0050] The gun-ball linkage in the gun-ball linkage system usually relies on the gun-ball coordinate calibration, that is, the gun-ball linkage relies on the coordinate mapping relationship between the gun camera and the ball camera obtained by calibration. After the gun-ball coordinate calibration is completed, the ball camera tracks the monitoring screen of the gun camera according to the pre-calibrated coordinate mapping relationship. However, the inventor found that for the gun-ball linkage system with adjustable gun pitch angle, after completing the gun-ball coordinate calibration, if the pitch angle of the gun camera lens changes, the coordinate relationship between the gun camera and the ball camera will change, and the ball camera will not track accurately, resulting in low accuracy of screen tracking.

[0051] The gun-ball linkage system control method provided in the embodiment of the present application can be applied to Figure 1 In the gun-ball linkage system shown, the gun-ball linkage system includes a control device, a gun camera and a ball camera, wherein the gun camera and the ball camera terminal device communicate with the control device through a network or a cable. The gun camera is a gun camera with an adjustable pitch angle.

[0052] During the operation of the gun-ball linkage system, the control device is used to: perform coordinate correlation calibration on the screen pixels of the gun camera and the ball camera, and determine the initial coordinate mapping relationship between the gun camera and the ball camera under the calibrated pitch angle; after the coordinate correlation calibration, detect the pitch angle of the gun camera during operation, and when it is detected that the pitch angle of the gun camera has changed, correct the calibrated initial coordinate mapping relationship between the gun camera and the ball camera according to the change of the pitch angle of the gun camera, and determine the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera; according to the corrected gun-ball coordinate mapping relationship, control the ball camera to perform linkage tracking on the monitoring screen of the gun camera.

[0053] In the present embodiment, after the coordinate correlation calibration is performed on the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, the pitch angle of the gun camera during operation is detected. When the pitch angle of the gun camera changes, the coordinate mapping relationship between the gun camera and the ball camera can be corrected in real time according to the change of the pitch angle of the gun camera, thereby reducing the screen pixel coordinate mapping error caused by the change of the gun camera angle, and reducing the situation where the ball camera tracking is inaccurate due to the change of the gun camera lens angle, so that the ball camera can quickly and accurately track the monitoring screen of the gun camera, thereby ensuring the accuracy of the gun-ball linkage and improving the accuracy of screen tracking.

[0054] In addition, compared with the method in the related art of reloading the images of the gun and ball camera to perform coordinate association calibration when the gun's elevation angle changes, the present solution directly corrects the calibrated initial coordinate mapping relationship according to the change in the gun angle. There is no need to reload the image to perform coordinate calibration every time the gun's elevation angle changes. This can reduce the tedious operations of the gun-ball coordinate association mapping, improve the efficiency and accuracy of the gun-ball coordinate mapping, improve the high precision, high efficiency and high reliability of the gun-ball collaborative work, and significantly improve the performance of the gun-ball linkage system.

[0055] The control device may be any type of electronic control unit, or any type of terminal device, including but not limited to any personal computer, laptop computer, smart phone, tablet computer, and portable wearable device. In other embodiments, the control device may also be a server, and the server may be implemented as an independent server or a server cluster composed of multiple servers.

[0056] In one embodiment, if Figure 2 As shown, a gun-ball linkage system control method is provided, and the method is applied in Figure 1 The control device in the example is used to illustrate, including the following steps:

[0057] S10: After the coordinate correlation calibration is performed on the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, the pitch angle of the gun camera during the working process is detected.

[0058] After the gun-ball linkage system is started, the control device performs coordinate correlation calibration on the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, and determines the initial coordinate mapping relationship between the gun camera and the ball camera under the calibration pitch angle. The calibration pitch angle is the pitch angle of the gun camera when the coordinate correlation calibration is performed.

[0059] Among them, after the gun-ball linkage system of the control device is started, the pitch angle of the gun camera is controlled to be a fixed pitch angle (i.e., a calibrated pitch angle), and then a specific calibration point (such as a fixed-position marker, laser point or artificial target) is selected in the monitoring screen to ensure that the calibration point appears in the screens of the gun camera and the ball camera at the same time; then the first pixel coordinate of the calibration point in the gun camera screen is determined, and the second pixel coordinate of the calibration point in the ball camera screen is determined, and the first pixel coordinate and the second pixel coordinate are recorded and stored as the coordinate pair data of the calibration point; the control device changes the calibration point, obtains multiple coordinate pair data of different calibration points under the calibrated pitch angle, and determines the initial coordinate mapping relationship between the gun camera and the ball camera under the calibrated pitch angle according to the multiple coordinate pair data of different calibration points. Among them, the initial coordinate mapping relationship may include multiple coordinate pair data of different calibration points; the initial coordinate mapping relationship may be stored in the form of a homography matrix H.

[0060] After the coordinate correlation calibration is performed on the image pixels of the gun camera and the ball camera and the initial coordinate mapping relationship between the gun camera and the ball camera at the calibrated pitch angle is determined, the control device controls the gun camera to monitor the image of the monitoring area and detects whether the pitch angle of the gun camera changes during the operation of the gun camera, that is, determines whether the pitch angle during the operation of the gun camera is the calibrated pitch angle.

[0061] Among them, the control device determines whether the current pitch angle of the gun is consistent with the calibrated pitch angle. If the current pitch angle of the gun is consistent with the calibrated pitch angle, it means that the gun lens has not rotated, and it is determined that no change in the pitch angle of the gun is detected; if the current pitch angle of the gun is inconsistent with the calibrated pitch angle, it means that the gun lens has rotated, and it is determined that a change in the pitch angle of the gun is detected. It is simple and convenient.

[0062] When no change in the elevation angle of the gun camera is detected, that is, the gun camera has not rotated and the current elevation angle of the gun camera is the calibrated elevation angle, there is no need to correct the calibrated initial coordinate mapping relationship between the gun camera and the ball camera. The control device controls the ball camera to perform linkage tracking on the monitoring screen of the gun camera based on the initial coordinate mapping relationship to ensure the accuracy of the ball camera screen tracking.

[0063] For example, the initial coordinate mapping relationship includes multiple coordinate pair data of different calibration points, and the coordinate pair data includes the coordinates of different calibration points in the gun camera and the coordinates of corresponding calibration points in the ball camera. When receiving the tracking instruction of the target point in the gun camera monitoring screen input by the user, the pixel coordinates of the target point in the ball camera are calculated by using the interpolation algorithm based on the multiple coordinate pair data of different calibration points in the initial coordinate mapping relationship, or the pixel coordinates of the target calibration point in the ball camera are calculated by using the H matrix mapping algorithm based on the gun-ball coordinate mapping relationship; the rotation value required by the ball camera (i.e., the rotation angle, including the horizontal rotation angle and the pitch angle) is calculated through the pixel coordinates of the target point in the ball camera, and a control signal is sent to drive the ball camera to rotate, so as to realize the manual tracking of the target in the gun camera monitoring screen. In other embodiments, an interpolation algorithm can be used to calculate the first pixel coordinate of the target point based on multiple coordinate pair data of different calibration points in the initial coordinate mapping relationship, and an H matrix mapping algorithm can be used to calculate the second pixel coordinate of the target point based on the initial coordinate mapping relationship. The optimal coordinates are determined from the first pixel coordinates and the second pixel coordinates as the pixel coordinates of the target point in the ball camera to calculate the rotation value required by the ball camera, and a control signal is sent to drive the ball camera to rotate, thereby realizing manual tracking of the target on the gun camera monitoring screen.

[0064] In the case of not receiving the tracking instruction of the target point, the control device performs target recognition on the monitoring screen of the gun camera. When the target to be tracked (such as a person or a car) is recognized, the H matrix mapping calculation is performed based on the pixel coordinates of the target to be tracked in the monitoring screen of the gun camera using the initial coordinate mapping relationship to determine the ball camera coordinates corresponding to the pixel coordinates of the target to be tracked in the monitoring screen of the gun camera (i.e., the pixel coordinates of the ball camera screen); or, based on the coordinate pair data of the calibration point in the initial coordinate mapping relationship, the ball camera coordinates corresponding to the pixel coordinates of the target to be tracked are calculated using the interpolation algorithm. Then, based on the ball camera coordinates, the required rotation value of the ball camera (i.e., the rotation angle, including the horizontal rotation angle and the pitch angle) is calculated, and a control signal is sent to drive the ball camera to rotate, so as to realize automatic tracking of the target in the monitoring screen of the gun camera.

[0065] S20: When a change in the pitch angle of the gun camera is detected, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the change in the pitch angle of the gun camera, and a corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined.

[0066] It should be understood that for a bolt with adjustable angle, in addition to the xy coordinate system of the imaging plane, the bolt's lens can adjust the viewing angle in the vertical direction, that is, the bolt's lens can rotate up and down in the vertical direction to better focus on the target or change the field of view. The change in the bolt's elevation angle usually increases from bottom to top. Taking the horizontal elevation angle of 0° as the reference angle, the maximum downward and upward rotation values ​​of the bolt's elevation angle can form the bolt's elevation angle range. Among them, the maximum downward and upward rotation values ​​of the bolt's elevation angle can be the same, that is, both are t max , then the elevation angle range of the bolt is [-t max ,t max ], that is, the maximum change of the gun's elevation angle is 2t max ,like Figure 3 shown.

[0067] During the operation of the bolt, the monitoring screen of the bolt at a certain elevation angle can be as follows Figure 4 The left side of the figure shows the screen A + screen B; after that, the bolt camera lens rotates upward to change the pitch angle. After the pitch angle of the bolt camera changes, the monitoring screen of the bolt camera can be as follows Figure 4 Picture A + Picture C shown on the right side of the figure. That is, Figure 4Picture A in the middle is the gun camera picture that can be seen before and after adjusting the gun camera's elevation angle, Picture B is the gun camera picture that is invisible after adjusting the gun camera's elevation angle, and Picture C is the gun camera monitoring picture added after adjusting the gun camera's elevation angle. During the operation of the gun-ball linkage system, the control device calibrates the coordinate association of the gun camera and the ball camera based on the monitoring picture A+B, and obtains the initial coordinate mapping relationship of the monitoring picture A+B based on the elevation angle; after completing the gun-ball coordinate association calibration, the ball camera tracks the gun camera's monitoring picture based on the initial coordinate mapping relationship. However, after the gun camera's elevation angle changes, the gun camera's monitoring picture becomes... Figure 4 The screen A+C causes part or all of the monitoring screen of the ball camera that originally tracked screen B to be outside the gun camera screen A+C. Errors in the ball camera tracking screen require recalibration of the coordinate mapping relationship.

[0068] In this regard, in this embodiment, when it is detected that the pitch angle of the gun camera has changed, that is, when the gun camera lens has rotated so that the current pitch angle of the gun camera is not the calibrated pitch angle, the control device corrects the calibrated initial coordinate mapping relationship between the gun camera and the ball camera according to the change in the pitch angle after the gun camera is rotated, and determines the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera.

[0069] Since the pitch angle of the gun camera changes in the vertical direction, its change will cause the coordinates of the pixel points of the ball camera to change. The displacement of the pixel points of the ball camera in different coordinate directions can be determined according to the difference between the pitch angle after the gun camera is rotated and the calibrated pitch angle. Then, according to the displacement of the pixel points of the ball camera in different coordinate directions, the coordinate mapping relationship of different pixel points in the initial coordinate mapping relationship is adjusted to obtain the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera.

[0070] S30: According to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera.

[0071] When a change in the elevation angle of the gun camera is detected, the control device controls the ball camera to perform linkage tracking on the monitoring screen of the gun camera according to the corrected gun-ball coordinate mapping relationship, so that the ball camera can accurately track the target tracking point in the monitoring screen of the gun camera.

[0072] In which, when the tracking instruction of the target calibration point in the gun camera monitoring screen input by the user is received, the pixel coordinates of the target calibration point in the ball camera are calculated by using an interpolation algorithm based on the multiple coordinate pair data of different calibration points in the revised gun-ball coordinate mapping relationship, and the pixel coordinates of the target calibration point in the ball camera are calculated by using an H matrix mapping algorithm based on the revised gun-ball coordinate mapping relationship; the rotation value (i.e., rotation angle, including horizontal rotation angle and pitch angle) required by the ball camera is calculated by using the pixel coordinates of the target calibration point in the ball camera, and a control signal is sent to drive the ball camera to rotate, so as to realize the manual tracking of the target in the gun camera monitoring screen. In other embodiments, the first pixel coordinates of the target point can also be calculated by using an interpolation algorithm based on the multiple coordinate pair data of different calibration points in the revised gun-ball coordinate mapping relationship, and coordinate mapping is performed based on the initial coordinate mapping relationship to obtain the second pixel coordinates of the target point, and the optimal coordinates are determined from the first pixel coordinates and the second pixel coordinates as the pixel coordinates of the target point in the ball camera to calculate the rotation value required by the ball camera, and a control signal is sent to drive the ball camera to rotate, so as to realize the manual tracking of the target in the gun camera monitoring screen.

[0073] In the case of not receiving the tracking instruction of the target calibration point, the control device performs target recognition on the monitoring screen of the gun camera. When the target to be tracked (such as a person or a car) is recognized, the H matrix mapping calculation is performed based on the pixel coordinates of the target to be tracked in the monitoring screen of the gun camera according to the corrected gun-ball coordinate mapping relationship to determine the ball camera coordinates (i.e., the pixel coordinates of the ball camera screen) corresponding to the pixel coordinates of the target to be tracked in the monitoring screen of the gun camera; or, based on the coordinate pair data of the calibration point in the initial coordinate mapping relationship, the ball camera coordinates corresponding to the pixel coordinates of the target to be tracked are calculated using an interpolation algorithm; then the required rotation value of the ball camera (i.e., the rotation angle, including the horizontal rotation angle and the pitch angle) is calculated according to the corresponding ball camera coordinates, and a control signal is sent to drive the ball camera to rotate, so as to realize automatic tracking of the target in the monitoring screen of the gun camera.

[0074] In this embodiment, when the gun's pitch angle changes, the coordinate mapping relationship between the gun and the ball camera can be corrected in real time according to the change of the gun's pitch angle, reducing the error in the coordinate mapping of the screen pixels caused by the change of the gun's angle, reducing the situation where the ball camera is not accurately tracking due to the change of the gun's lens angle, so that the ball camera can quickly and accurately track the monitoring screen of the gun, ensuring the accuracy of the gun-ball linkage, and improving the accuracy of the screen tracking. In addition, there is no need to reload the screen for coordinate calibration every time the gun's pitch angle changes, which can reduce the tedious operation of the gun-ball coordinate association mapping, improve the efficiency and accuracy of the gun-ball coordinate mapping, improve the high precision, high efficiency and high reliability of the gun-ball collaborative work, and significantly improve the performance of the gun-ball linkage system.

[0075] In one embodiment, if Figure 5As shown, in step S20, the initial coordinate mapping relationship between the calibrated gun camera and the ball camera is corrected according to the change of the pitch angle of the gun camera, which specifically includes the following steps:

[0076] S201: When it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the gun.

[0077] When a change in the pitch angle of the bolt is detected, the control device can determine whether the pixel range of the monitoring screen after the bolt is rotated is consistent with the pixel range of the calibration screen; wherein the calibration screen pixel range is the pixel range of the monitoring screen of the bolt during the coordinate association calibration. That is, when a change in the pitch angle of the bolt is detected, the control device can determine whether the pixel range of the monitoring screen after the bolt is rotated is consistent with the pixel range of the monitoring screen of the bolt during the coordinate association calibration.

[0078] When it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, it means that the monitoring screen resolution of the gun is consistent with the screen resolution during coordinate association calibration. The control device can directly correct the initial coordinate mapping relationship according to the change of the gun's pitch angle.

[0079] S202: When it is determined that the pixel range of the monitoring screen after the gun is rotated is inconsistent with the pixel range of the calibration screen, the pixel range of the monitoring screen after the gun is rotated is adjusted, and when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the gun's pitch angle.

[0080] When it is determined that the pixel range of the monitoring screen after the gun is rotated is inconsistent with the pixel range of the calibration screen, it means that the resolution of the monitoring screen of the gun is inconsistent with the screen resolution during the coordinate association calibration, and the resolution of the current monitoring screen needs to be adjusted to the screen resolution during the coordinate association calibration (i.e., the original resolution), that is, the screen resolution needs to be consistent to ensure the picture quality and the convenience and accuracy of the subsequent coordinate association mapping. Among them, the control device can perform pixel sampling on the monitoring screen after the gun is rotated until the resolution of the monitoring screen of the gun is adjusted to the screen resolution during the coordinate association calibration, that is, the pixel range of the monitoring screen after the gun is rotated is sampled to be consistent with the pixel range of the monitoring screen of the gun during the coordinate association calibration. Among them, by performing pixel sampling on the monitoring screen after the gun is rotated, the pixel coordinates in the gun monitoring screen can be normalized and converted, so that the resolution of the normalized gun monitoring screen is consistent with the screen resolution during the coordinate association calibration.

[0081] When it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, that is, when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the monitoring screen of the gun during coordinate association calibration, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the gun.

[0082] In this embodiment, when it is detected that the pitch angle of the gun has changed, if it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the gun; if the pixel range of the monitoring screen after the gun is rotated is inconsistent with the pixel range of the calibration screen, the pixel range of the monitoring screen after the gun is rotated is adjusted, and when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the gun, which can ensure the picture quality and improve the convenience and accuracy of subsequent coordinate association mapping.

[0083] In one embodiment, if Figure 6 As shown, in step S20 (or steps S201 and S202), the initial coordinate mapping relationship between the calibrated gun camera and the ball camera is corrected according to the change of the pitch angle of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined, which specifically includes the following steps:

[0084] S21: Determine the displacement proportional coefficient of the bolt according to the change in the elevation angle after the bolt is rotated.

[0085] When the change in the pitch angle of the bolt is detected, the control device determines the displacement proportional coefficient of the bolt according to the change in the pitch angle after the bolt is rotated. The displacement proportional coefficient is the ratio of the change in the pitch angle after the bolt is rotated to the maximum change in the pitch angle of the bolt. The displacement proportional coefficient is determined by the change in the pitch angle after the bolt is rotated, so as to measure the relative displacement of each pixel in the monitoring screen after the bolt is rotated.

[0086] Among them, when it is detected that the elevation angle of the bolt has changed, the control device can determine the displacement proportional coefficient of the bolt according to the difference between the elevation angle after the bolt is rotated and the calibrated elevation angle.

[0087] S22: According to the displacement proportional coefficient of the bolt, the coordinate correction data of the ball camera after the bolt rotates is determined.

[0088] After determining the displacement proportional coefficient of the bolt, the control device determines the coordinate correction data of the ball camera after the bolt is rotated according to the displacement proportional coefficient of the bolt.

[0089] The coordinate correction data of the ball camera may be a constant value that changes with the pitch angle of the gun, and the constant value is determined according to the displacement proportionality coefficient and a fixed ball camera correction value. For example, a pre-calibrated ball camera correction value may be obtained, and then the displacement proportionality coefficient may be multiplied by the pre-calibrated ball camera correction value to obtain a constant value that changes with the pitch angle of the gun, as the coordinate correction data of the ball camera after the gun is rotated. In other embodiments, the coordinate correction data of the ball camera may also be a set of coordinate correction values ​​of different calibration points, that is, the coordinate correction data of the ball camera may include the coordinate correction values ​​required for different calibration points in the ball camera after the gun is rotated.

[0090] S23: Correct the initial coordinate mapping relationship according to the coordinate correction data of the ball camera to obtain a corrected gun-ball coordinate mapping relationship.

[0091] After determining the coordinate correction data of the ball camera after the gun is rotated, the control device corrects the initial coordinate mapping relationship according to the coordinate correction data of the ball camera to obtain a corrected gun-ball coordinate mapping relationship.

[0092] For example, the control device can determine the initial coordinates of multiple calibration points of the ball camera in the initial coordinate mapping relationship; subtract the coordinate correction value required for the corresponding calibration point in the coordinate correction data from the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship to obtain the corrected gun-ball coordinate mapping relationship.

[0093] Among them, taking the coordinate correction data of the ball camera as a constant value as an example, the coordinates of the calibration point of the ball camera in the corrected gun-ball coordinate mapping relationship can be expressed by the following formula:

[0094] PT T1 =PT T -α*ΔT′;

[0095] Wherein, PT represents the preset calibration point; PT T Represents the initial coordinates of the calibration point PT in the initial coordinate mapping relationship; PT T1 It represents the coordinates of the calibration point PT in the corrected gun-ball coordinate mapping relationship; α represents the displacement proportional coefficient of the gun; α*ΔT′ represents the coordinate correction data of the ball camera; ΔT′ represents the pre-calibrated ball camera correction value, that is, after the gun is rotated according to the maximum change value of the pitch angle, the coordinate change value of the ball camera corresponding to the calibration point whose x coordinate (i.e. horizontal coordinate) is located in the middle of the picture, which is a constant value.

[0096] Among them, the calculation formula for the coordinates of the calibration points of the ball camera in the corrected coordinate mapping relationship is only for exemplary explanation. In other embodiments, the calculation formula for the coordinates of the calibration points of the ball camera in the corrected coordinate mapping relationship can also be other, which will not be repeated here.

[0097] In this embodiment, the displacement proportional coefficient of the bolt is determined according to the change of the pitch angle after the bolt is rotated, and the displacement proportional coefficient is the relative displacement of the pitch angle relative to the pitch angle range of the bolt after the bolt is rotated; according to the displacement proportional coefficient, the coordinate correction data of the ball camera after the bolt is rotated is determined, and then the initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain the corrected gun-ball coordinate mapping relationship. The displacement proportional coefficient can directly reflect the position offset of the target point in the bolt screen caused by the change of the pitch angle, and then according to the displacement proportional coefficient of the bolt, the coordinate offset that needs to be corrected in the ball camera screen is calculated, and the initial coordinate mapping relationship is corrected. The coordinate mapping consistency of the bolt and the ball camera can be maintained, so that the coordinate mapping relationship between the bolt and the ball camera can be updated in real time, ensuring the accuracy of linkage monitoring.

[0098] In one embodiment, step S21, that is, determining the displacement proportional coefficient of the bolt according to the change of the pitch angle after the bolt is rotated, the displacement proportional coefficient is the relative displacement of the pitch angle relative to the pitch angle range of the bolt after the bolt is rotated, specifically includes the following steps:

[0099] S211: Determine the difference between the pitch angle after the bolt is rotated and the calibrated pitch angle to obtain a pitch angle change value.

[0100] When the elevation angle of the bolt is detected to have changed, the control device obtains the elevation angle change value according to the difference between the elevation angle after the bolt is rotated and the calibrated elevation angle, wherein the calibrated elevation angle is the elevation angle of the bolt when the coordinate association calibration is performed.

[0101] S212: Determine the maximum variation value of the pitch angle of the bolt according to the pitch angle range of the bolt.

[0102] At the same time, the control device determines the maximum change value of the elevation angle of the bolt according to the elevation angle range of the bolt. Figure 3 As shown, the elevation angle range of the bolt is [-t max ,t max ], that is, the maximum change of the gun's elevation angle is 2t max .

[0103] S213: Determine the ratio of the pitch angle change value to the maximum pitch angle change value as the displacement proportional coefficient of the bolt.

[0104] After determining the pitch angle change value of the pitch angle after the bolt is rotated and the calibrated pitch angle, the control device determines the ratio of the pitch angle change value to the maximum pitch angle change value as the displacement proportional coefficient of the bolt. Among them, the displacement proportional coefficient is the direct ratio of the pitch angle difference of the bolt relative to the calibrated pitch angle after the coordinate association calibration is completed, and the maximum pitch angle change value within the bolt pitch angle range. By determining the difference between the actual pitch angle of the bolt and the calibrated pitch angle, the relative displacement relative to the bolt pitch angle range is obtained to obtain the displacement proportional coefficient, which is used to measure the relative displacement of each pixel point in the bolt screen, thereby improving the accuracy of the displacement proportional coefficient.

[0105] Among them, the displacement proportional coefficient of the bolt is determined by the following formula:

[0106]

[0107] Among them, α represents the displacement proportional coefficient of the bolt; Δt represents the difference between the pitch angle after the bolt is rotated and the calibrated pitch angle, that is, the pitch angle change value; 2t max Indicates the maximum change value of the gun's elevation angle. max Indicates the maximum value to which the bolt's elevation angle can be rotated upward or downward.

[0108] In other embodiments, the displacement proportional coefficient of the bolt can also be determined according to the difference between the pitch angle of the bolt after the bolt is rotated and the pitch angle of the bolt before the pitch angle changes. For example, after completing the coordinate association calibration and determining the initial coordinate mapping relationship between the bolt and the ball camera at the calibrated pitch angle, the bolt rotates, and two pitch angle changes are detected during the rotation process. When the pitch angle change is detected for the first time, the displacement proportional coefficient of the bolt is determined according to the difference between the pitch angle of the bolt when the pitch angle change is detected for the first time and the calibrated pitch angle. When the pitch angle change is detected for the second time, the displacement proportional coefficient of the bolt is determined according to the difference between the pitch angle of the bolt when the pitch angle change is detected for the second time and the pitch angle of the bolt when the pitch angle change is detected for the first time. In this way, the last pitch angle is used as the difference calculation standard. When the pitch angle changes for multiple times, errors may be superimposed, thereby increasing the correction error of the subsequent initial coordinate mapping relationship.

[0109] In the present embodiment, when two changes in the pitch angle are detected, the displacement proportional coefficient of the bolt can be determined according to the difference between the pitch angle of the bolt when the pitch angle change is detected and the calibrated pitch angle; each time a change in the pitch angle is detected, the calibrated pitch angle is used as a reference standard, which can reduce the error in the subsequent correction of the initial coordinate mapping relationship and improve the accuracy of the linkage tracking.

[0110] In this embodiment, the difference between the pitch angle after the bolt is rotated and the calibrated pitch angle is determined to obtain the pitch angle change value; the maximum pitch angle change value of the bolt is determined according to the pitch angle range of the bolt; the ratio of the pitch angle change value to the maximum pitch angle change value is determined as the displacement proportional coefficient of the bolt. In this solution, each time a change in the pitch angle is detected, the calibrated pitch angle is used as a reference standard, which can reduce the error of the subsequent correction of the initial coordinate mapping relationship and improve the accuracy of linkage monitoring.

[0111] In one embodiment, in step S22, that is, determining the coordinate correction data of the ball camera after the bolt rotates according to the displacement proportional coefficient, specifically includes the following steps:

[0112] S221: Determine the calibration point adjustment data of the pre-calibrated ball camera.

[0113] After determining the displacement proportional coefficient of the gun camera, the control device needs to obtain the calibration point adjustment data of the pre-calibrated ball camera. The calibration point adjustment data of the ball camera includes the coordinate displacement values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera is rotated according to the maximum change value of the pitch angle. In one embodiment, the calibration point adjustment data of the ball camera includes the coordinate displacement values ​​required for multiple calibration points in the ball camera screen to always point to the specified pixel coordinates (such as the specified calibration points) in the gun camera screen after the gun camera is rotated according to the maximum change value of the pitch angle.

[0114] It is important to understand that when the bolt camera rotates and the ball camera tracks the bolt camera, the bolt camera's pitch angle changes, and the coordinates of the pixel points in the bolt camera screen change, that is, the coordinates of the calibration points in the bolt camera screen change. Figure 7 As shown, when the elevation angle of the bolt lens is -t max When the elevation angle is set, the coordinate positions of multiple calibration points (indicated by crosses in the figure) on the bolt screen refer to Figure 7 The cross-shaped position of the upper rectangular frame. max The pitch angle position moves to t max When the elevation angle is set to the desired position, the coordinate positions of multiple calibration points on the bolt screen move downward, and the coordinate displacement value of each calibration point is Δ. Figure 7 The cross position of the lower middle rectangular box changes. Figure 7 The dotted cross in the lower rectangle indicates that the bolt camera's elevation angle is -t max The coordinate position of the calibration point at the time of position; Figure 7 The solid cross in the rectangular box below indicates that the gun lens pitch angle is t max The coordinate position of the calibration point at the time of the position, Δ represents the coordinate displacement value of the calibration point. That is, the pitch angle of the bolt lens changes from -t max Position rotated to t maxWhen the position is changed, the displacement value of any calibration point moving downward is Δ, and the displacement values ​​of calibration points with different x-coordinates (i.e., horizontal axis coordinates) are different. As the gun camera pitch angle changes, the coordinates of the calibration points in the gun camera screen change (the pixel coordinates change). To ensure the accuracy of the ball camera tracking, the coordinate positions of the calibration points in the ball camera screen also need to be adjusted accordingly to calibrate the calibration point adjustment data of the ball camera.

[0115] The calibration process of the calibration point adjustment data of the ball camera is as follows: control the gun to rotate according to the maximum change value of the pitch angle, that is, control the pitch angle of the gun from t max The pitch angle position is rotated to -t max The pitch angle position (or from -t max The pitch angle position is rotated to t max The pitch angle position of the ball camera is synchronously controlled to rotate to track the monitoring screen of the gun camera, and the coordinate displacement values ​​of the top group of calibration points on the ball camera screen (in the vertical direction) and the coordinate displacement values ​​of the bottom group of calibration points on the ball camera screen (in the vertical direction) are recorded; the coordinate displacement values ​​of the two groups of calibration points are interpolated to obtain the coordinate displacement values ​​of multiple calibration points on the ball camera screen, that is, the calibration point adjustment data of the ball camera is obtained.

[0116] S222: Multiply the displacement proportional coefficient by the calibration point adjustment data to determine the coordinate correction data of the ball camera after the bolt is rotated.

[0117] The control device multiplies the displacement proportional coefficient with the calibration point adjustment data to determine the coordinate correction data of the ball camera after the gun rotates. The coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun rotates.

[0118] It should be understood that when the bolt rotates and the ball camera tracks the rotation, the pitch angle of the bolt changes. At the same pitch angle change value, the coordinate displacement of different pixel points in the ball camera screen is not a straight line, but a curved line. That is, at the same pitch angle change value, the coordinate displacement values ​​that need to be associated and adjusted in the pixel points with different x coordinates in the ball camera screen are different. In order to ensure the accuracy of the subsequent coordinate mapping relationship correction and ensure that the calibration points are evenly distributed in the screen, different coordinate displacement values ​​can be calibrated for different calibration points (pixel points), and the coordinate displacement values ​​ΔT corresponding to calibration points with different x coordinates are different. That is, each calibration point in the calibration point adjustment data of the ball camera corresponds to a coordinate displacement value that needs to be associated and adjusted, and the coordinate displacement values ​​corresponding to calibration points with different x coordinates are different.

[0119] In this embodiment, the calibration point adjustment data of the pre-calibrated ball camera is determined, and the calibration point adjustment data includes the coordinate displacement values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera rotates according to the maximum change value of the pitch angle, and the coordinate displacement values ​​corresponding to the calibration points with different horizontal coordinates are different; the displacement ratio coefficient is multiplied by the calibration point adjustment data to determine the coordinate correction data of the ball camera after the gun camera rotates, and the coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera rotates. In this scheme, different coordinate displacement values ​​are calibrated for different pixel points in different pictures of the ball camera to obtain the calibration point adjustment data of the ball camera. The accurate coordinate correction data of the ball camera can be determined according to the calibration point adjustment data of the ball camera and the displacement ratio coefficient. The initial coordinate mapping relationship is corrected by the coordinate correction data, which improves the accuracy of the corrected gun-ball coordinate mapping relationship, so that after the gun camera pitch angle changes, multiple calibration points of the ball camera can still be evenly distributed in the monitoring picture, thereby improving the accuracy of the gun-ball linkage monitoring.

[0120] In one embodiment, the coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera rotates. In step S23, the initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain the corrected gun-ball coordinate mapping relationship, which specifically includes the following steps:

[0121] S231: Determine the initial coordinates of multiple calibration points of the ball camera in the initial coordinate mapping relationship.

[0122] S232: Subtract the coordinate correction value required for the associated adjustment of the corresponding calibration point in the coordinate correction data from the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship to obtain a corrected gun-ball coordinate mapping relationship.

[0123] In this embodiment, the coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun is rotated. The control device can determine the initial coordinates of multiple calibration points of the ball camera in the initial coordinate mapping relationship, and then subtract the coordinate correction values ​​required for the associated adjustment of the corresponding calibration points in the coordinate correction data from the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship to obtain the corrected gun-ball coordinate mapping relationship.

[0124] Among them, the coordinates of the calibration point of the ball camera in the corrected gun-ball coordinate mapping relationship can be expressed by the following formula:

[0125] PT T1 =PT T -α*ΔT;

[0126] Wherein, PT represents a preset calibration point; PT T Represents the initial coordinates of the calibration point PT in the initial coordinate mapping relationship; PT T1It represents the coordinates of the calibration point PT in the corrected gun-ball coordinate mapping relationship; α represents the displacement proportional coefficient of the gun; α*ΔT represents the coordinate correction value required for the associated adjustment of the calibration point PT in the coordinate correction data of the ball camera; ΔT represents the coordinate displacement value required for the associated adjustment of the calibration point PT in the ball camera after the gun is rotated according to the maximum change value of the pitch angle in the calibration point adjustment data.

[0127] In the present embodiment, the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship are determined; the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship are subtracted from the coordinate correction values ​​required for the associated adjustment of different calibration points in the coordinate correction data, so as to obtain a corrected gun-ball coordinate mapping relationship; the coordinate correction values ​​of different calibration points in the ball camera screen after the pitch angle changes during the rotation of the gun are determined through the calibration point adjustment data, so as to correct the pixel coordinates of the ball camera at different calibration points, thereby improving the accuracy of the corrected gun-ball coordinate mapping relationship, so that the multiple calibration points of the ball camera can still be evenly distributed in the monitoring screen after the pitch angle of the gun changes, thereby improving the accuracy of the gun-ball linkage monitoring.

[0128] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] In one embodiment, a control device is provided, which corresponds one-to-one with the gun-ball linkage system control method in the above embodiment. Figure 8 As shown, the control device includes a detection module 801, a correction module 802 and a control module 803. The detailed description of each functional module is as follows:

[0130] The detection module 801 is used to detect the pitch angle of the gun camera during operation after the coordinate correlation calibration of the screen pixels of the gun camera and the ball camera;

[0131] The correction module 802 is used to correct the initial coordinate mapping relationship between the calibrated gun camera and the ball camera according to the change of the gun camera's pitch angle when the gun camera's pitch angle changes, and determine the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera;

[0132] The control module 803 is used to control the ball camera to perform linkage tracking on the monitoring screen of the gun camera according to the corrected gun-ball coordinate mapping relationship.

[0133] Optionally, the correction module 802 is specifically used for:

[0134] According to the change of the elevation angle after the bolt is rotated, the displacement proportional coefficient of the bolt is determined, and the displacement proportional coefficient is the relative displacement of the elevation angle relative to the elevation angle range of the bolt after the bolt is rotated;

[0135] According to the displacement proportional coefficient, the coordinate correction data of the ball camera after the bolt rotates is determined;

[0136] The initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain a corrected gun-ball coordinate mapping relationship.

[0137] Optionally, the correction module 802 is further configured to:

[0138] Determine the difference between the elevation angle after the bolt is rotated and the calibrated elevation angle to obtain the elevation angle change value;

[0139] Determine the maximum change value of the elevation angle of the bolt according to the elevation angle range of the bolt;

[0140] The ratio of the pitch angle change value to the maximum pitch angle change value is determined as the displacement proportional coefficient of the bolt.

[0141] Optionally, the root correction module 802 is further configured to:

[0142] Determine the calibration point adjustment data of the pre-calibrated ball camera, the calibration point adjustment data including the coordinate displacement values ​​required for the associated adjustment of different calibration points in the ball camera after the gun camera rotates according to the maximum change value of the pitch angle;

[0143] The displacement proportional coefficient is multiplied by the calibration point adjustment data to determine the coordinate correction data of the ball camera after the gun rotates. The coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun rotates.

[0144] Optionally, the coordinate correction data includes coordinate correction values ​​required for associated adjustment of different calibration points in the ball camera after the bolt rotates, and the correction module 802 is further specifically used for:

[0145] Determine the initial coordinates of multiple calibration points of the ball camera in the initial coordinate mapping relationship;

[0146] The initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship are subtracted from the coordinate correction values ​​required for the associated adjustment of different calibration points in the coordinate correction data to obtain the corrected gun-ball coordinate mapping relationship.

[0147] Optionally, the control device further includes a sampling module 804, and the correction module 802 is further specifically used to correct the initial coordinate mapping relationship according to the change of the pitch angle of the gun when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, and the pixel range of the calibration screen is the pixel range of the monitoring screen of the gun during the coordinate association calibration;

[0148] The sampling module 804 is specifically used to adjust the pixel range of the monitoring picture after the gun is rotated when it is determined that the pixel range of the monitoring picture after the gun is rotated is inconsistent with the pixel range of the calibration picture, so that the correction module 802 can correct the initial coordinate mapping relationship according to the change of the pitch angle of the gun when it is determined that the pixel range of the monitoring picture after the gun is rotated is consistent with the pixel range of the calibration picture.

[0149] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0150] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0151] The present application also provides an electronic device, such as Fig. 9 As shown, the electronic device 2 includes: at least one processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, wherein the processor 20 implements the steps in any of the above-mentioned method embodiments when executing the computer program 22, or implements the functions of each module / unit in the above-mentioned device embodiments when executing the computer program 22.

[0152] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0153] Those skilled in the art will understand that Fig. 9These are merely examples of the electronic device and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0154] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0155] The memory may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory may also include both an internal storage unit of the electronic device and an external storage device.

[0156] An embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0157] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0159] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0160] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0161] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A gun-ball linkage system control method, characterized in that: include: After the coordinate correlation calibration is performed on the screen pixels of the gun camera and the ball camera in the gun-ball linkage system, the pitch angle of the gun camera during operation is detected; When it is detected that the elevation angle of the gun camera changes, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the elevation angle change of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined; According to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera.

2. The gun-ball linkage system control method according to claim 1, characterized in that: The method of correcting the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration according to the change of the pitch angle of the gun camera and determining the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera includes: Determine a displacement proportional coefficient of the bolt according to a change in the pitch angle after the bolt is rotated, wherein the displacement proportional coefficient is a ratio of a change in the pitch angle after the bolt is rotated to a maximum change in the pitch angle of the bolt; Determining coordinate correction data of the ball camera after the bolt rotates according to the displacement proportional coefficient; The initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain the corrected gun-ball coordinate mapping relationship.

3. The gun-ball linkage system control method according to claim 2, characterized in that: The step of determining the displacement proportional coefficient of the bolt according to the change of the pitch angle after the bolt is rotated comprises: Determine the difference between the pitch angle after the bolt is rotated and the calibrated pitch angle to obtain a pitch angle change value, wherein the calibrated pitch angle is the pitch angle of the bolt when performing coordinate association calibration; Determining a maximum variation value of the pitch angle of the bolt according to the pitch angle range of the bolt; The ratio of the pitch angle change value to the maximum pitch angle change value is determined as the displacement proportional coefficient of the bolt.

4. The gun-ball linkage system control method according to claim 2, characterized in that: The step of determining the coordinate correction data of the ball camera after the gun rotates according to the displacement proportional coefficient includes: Determine the calibration point adjustment data of the pre-calibrated ball camera, wherein the calibration point adjustment data includes coordinate displacement values ​​required for associated adjustment of different calibration points in the ball camera after the gun camera rotates according to the maximum change value of the pitch angle; The displacement proportional coefficient is multiplied by the calibration point adjustment data to determine the coordinate correction data of the ball camera after the gun is rotated, and the coordinate correction data of the ball camera includes the coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun is rotated.

5. The gun-ball linkage system control method according to claim 2, characterized in that: The coordinate correction data includes coordinate correction values ​​required for the associated adjustment of different calibration points in the ball camera after the gun is rotated, and the initial coordinate mapping relationship is corrected according to the coordinate correction data of the ball camera to obtain the corrected gun-ball coordinate mapping relationship, including: Determine the initial coordinates of a plurality of calibration points of the ball camera in the initial coordinate mapping relationship; The coordinate correction values ​​required for the associated adjustment of different calibration points in the coordinate correction data are subtracted from the initial coordinates of the multiple calibration points of the ball camera in the initial coordinate mapping relationship to obtain the corrected gun-ball coordinate mapping relationship.

6. The gun-ball linkage system control method according to any one of claims 1 to 5, characterized in that: The correcting of the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration includes: When it is determined that the pixel range of the monitoring screen after the bolt is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the bolt, and the pixel range of the calibration screen is the pixel range of the monitoring screen of the bolt during the coordinate association calibration; When it is determined that the pixel range of the monitoring screen after the gun is rotated is inconsistent with the pixel range of the calibration screen, the pixel range of the monitoring screen after the gun is rotated is adjusted, and when it is determined that the pixel range of the monitoring screen after the gun is rotated is consistent with the pixel range of the calibration screen, the initial coordinate mapping relationship is corrected according to the change of the pitch angle of the gun.

7. A gun-ball linkage system, characterized in that: It includes a control device, a gun camera and a ball camera, wherein the control device is used for: After the coordinate correlation calibration is performed on the screen pixels of the gun camera and the ball camera, the pitch angle of the gun camera during operation is detected; When it is detected that the elevation angle of the gun camera changes, the initial coordinate mapping relationship between the gun camera and the ball camera obtained by calibration is corrected according to the elevation angle change of the gun camera, and the corrected gun-ball coordinate mapping relationship between the gun camera and the ball camera is determined; According to the corrected gun-ball coordinate mapping relationship, the ball camera is controlled to perform linkage tracking on the monitoring screen of the gun camera.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the gun-ball linkage system control method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the gun-ball linkage system control method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the gun-ball linkage system control method according to any one of claims 1 to 6.

Images