Optimization method and device based on gun and ball linkage monitoring system, medium and product

Abstract

The invention provides an optimization method and device based on a gun-dome camera linkage monitoring system, a medium and a product, and the method comprises the steps: obtaining N full-view target frames and K amplified target frames for an mth frame of gun camera video and an mth frame of dome camera video, and carrying out the preliminary matching and supplementary matching of the kth amplified target frame and the N full-view target frames through a cross-scale ID matching model; according to the results of the preliminary matching and the supplementary matching, obtaining a full-view target frame and an amplified target frame which have a corresponding relation and missing detection target data of the gun camera; and storing a full-view target frame and an amplified target frame which have a corresponding relation into a first m frame queue, and storing missing detection target data of a gun camera into a training data set of a small target detection model and a training data set of a cross-scale ID matching model so as to update the small target detection model and the cross-scale ID matching model. According to the invention, the problem that target missing detection and tracking missing detection are easy to occur in a gun-dome camera linkage monitoring system in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an optimization method, equipment, medium and product based on a gun-ball linkage monitoring system. Background Art

[0002] The gun-ball linkage monitoring system is widely used in various industries. It monitors the monitoring area by using a gun camera with a wide viewing angle. The targets in the monitoring area include large targets and small targets, such as pedestrians, vehicles, and non-motor vehicles. For these small targets, small target detection and other methods are needed to assist in identification; then the small targets in the monitoring area of ​​the gun camera are identified, tracked and magnified through the ball camera with a local viewing angle.

[0003] However, this gun-ball linkage monitoring system is highly dependent on the detection algorithm of the gun camera, and is prone to missed targets and missed tracks. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an optimization method based on a gun-ball linkage monitoring system, which solves the problem that the gun-ball linkage monitoring system in the prior art is prone to target missed detection and tracking missed detection.

[0005] According to an embodiment of the present invention, a first aspect provides an optimization method based on a gun-ball linkage monitoring system, comprising:

[0006] Obtain N panoramic target frames, where the N panoramic target frames are obtained by taking M frames of gun camera video and processing each frame of the gun camera video through a small target detection model;

[0007] Obtain K magnified target frames, where the K magnified target frames are obtained by taking M frames of dome camera videos and processing each frame of dome camera videos through a dome camera detection model; wherein the gun camera video and the dome camera video are videos taken by the gun camera and the dome camera based on the same target;

[0008] For the mth frame of the gun camera video and the mth frame of the ball camera video, the kth magnified target frame and the N full-view target frames are preliminarily matched and supplemented through the cross-scale ID matching model;

[0009] According to the results of the preliminary matching and the supplementary matching, the corresponding panoramic target frame and the enlarged target frame as well as the missed target data of the gun camera are obtained;

[0010] The corresponding overview target frame and the enlarged target frame are stored in the first m frame queue, and the missed target data of the gun camera are stored in the training data set of the small target detection model and the training data set of the cross-scale ID matching model to update the small target detection model and the cross-scale ID matching model;

[0011] N, K, and M are positive integers, k is a positive integer less than or equal to K, and m is a positive integer less than or equal to M.

[0012] Optionally, for the mth frame of the gun camera video and the mth frame of the ball camera video, the kth magnified target frame and the N panoramic target frames are preliminarily matched and supplementarily matched through a cross-scale ID matching model, including:

[0013] If the k-th enlarged target frame has the same ID as any one of the N overview target frames, the k-th enlarged target frame and the overview target frame with the same ID are preliminarily matched successfully, and the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame are the overview target frame and enlarged target frame with a corresponding relationship;

[0014] If the k-th enlarged target frame does not have the same ID as any of the N panoramic target frames, the preliminary matching of the k-th enlarged target frame and the N panoramic target frames fails, and the k-th enlarged target frame is supplementarily matched with all the panoramic target frames of the previous m frames of the gun camera video; wherein, the k-th enlarged target frame successfully obtained by the supplementary matching and the panoramic target frame with the same ID as the k-th enlarged target frame in the previous m frames of the gun camera video are the panoramic target frame and enlarged target frame with a corresponding relationship; the ID data of the k-th enlarged target frame obtained by the supplementary matching failure is the missed target data of the gun camera.

[0015] Optionally, the k-th enlarged target frame is supplementarily matched with all the panoramic target frames of the previous m frames of the gun camera video, including:

[0016] If the first m frames of gun camera video include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame in the first m frames of gun camera video are successfully supplemented and matched; if the first m frames of gun camera video do not include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame is supplemented and matched unsuccessfully, and the ID data of the k-th enlarged target frame is obtained.

[0017] Optionally, in the first m frame queues, the information stored in each frame queue includes: a global unified ID, an overview target frame ID, an enlarged target frame ID, a gun camera video image, and a ball camera video image.

[0018] Optionally, if the mth frame only includes the kth enlarged target frame, the global unified ID and the overview target frame ID are empty.

[0019] Optionally, when the training data set of the small target detection model is greater than a first preset number of images, updating the small target detection model;

[0020] When the training data set of the cross-scale ID matching model is greater than a second preset number of images, the cross-scale ID matching model is updated.

[0021] Optionally, for the mth frame of the gun camera video and the mth frame of the ball camera video, before the kth magnified target frame and the N panoramic target frames are preliminarily matched and supplementarily matched by the cross-scale ID matching model, the following steps are included:

[0022] Calculate the image mapping relationship between the mth frame of the gun camera video and the mth frame of the ball camera video, and transform the image point coordinates of the mth frame of the ball camera video;

[0023] The IDs of the K enlarged target frames are represented by the converted image point coordinates of the m-th frame of the ball camera video, and the IDs of the N panoramic target frames are represented by the image point coordinates of the m-th frame of the gun camera video.

[0024] The second aspect provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the optimization method based on the gun-ball linkage monitoring system as described in any one of claims 1 to 7 is implemented.

[0025] The third aspect provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the optimization method based on the gun-ball linkage monitoring system as described in any one of the above items.

[0026] A fourth aspect provides a computer program product, comprising a computer program, which, when executed by a processor, implements the optimization method based on the gun-ball linkage monitoring system as described in any one of the above items.

[0027] Compared with the prior art, the present invention has the following beneficial effects: for the video obtained by shooting the same target by the gun camera and the ball camera, i.e., cross-scale ID matching is performed frame by frame for the gun camera video and the ball camera video, so as to determine whether the panoramic target frame provided by the gun camera and the magnified target frame provided by the ball camera match in each frame of the gun camera video and the ball camera video, so as to determine whether the detection algorithm used by the gun camera, such as the small target detection model, has missed detection of the target and missed detection of tracking. If missed detection of the target and missed detection of tracking occur, the missed detection target data of the gun camera is stored in the training data set of the small target detection model and the training data set of the cross-scale ID matching model to update the small target detection model and the cross-scale ID matching model, so as to reduce the missed detection probability, and at the same time ensure that the ball camera can continue to track and magnify the panoramic target obtained by the gun camera. In addition, the present invention provides a two-time matching mechanism, i.e., preliminary matching and supplementary matching, to ensure the validity and availability of the missed detection target data of the gun camera and improve the optimization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the implementation flow of the optimization method based on the gun-ball linkage monitoring system according to an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of the composition structure of the optimization device based on the gun-ball linkage monitoring system according to an embodiment of the present invention;

[0030] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0032] like Figure 1 As shown, an embodiment of the present invention proposes an optimization method based on a gun-ball linkage monitoring system, which is used to improve the small target detection model of the gun camera, reduce the probability of missed detection, and improve the cross-scale ID matching model to ensure that the ball camera can continue to track and zoom in on the full-view target obtained by the gun camera.

[0033] The optimization method based on the gun-ball linkage monitoring system includes but is not limited to the following steps:

[0034] S1. Obtain N panoramic target frames, where the N panoramic target frames are obtained by taking M frames of gun camera videos and processing each frame of the gun camera videos through a small target detection model;

[0035] S2. Obtain K magnified target frames, where the K magnified target frames are obtained by taking M frames of dome camera videos and processing each frame of dome camera videos through a dome camera detection model; wherein the gun camera video and the dome camera video are videos taken by the gun camera and the dome camera based on the same target;

[0036] S3, for the mth frame of the gun camera video and the mth frame of the ball camera video, perform preliminary matching and supplementary matching on the kth magnified target frame and the N full-view target frames through the cross-scale ID matching model;

[0037] S4. According to the results of the preliminary matching and the supplementary matching, the panoramic target frame and the enlarged target frame having a corresponding relationship and the missed target data of the gun camera are obtained;

[0038] S5, storing the corresponding panoramic target frame and the enlarged target frame into the first m frame queue, storing the missed target data of the gun camera into the training data set of the small target detection model and the training data set of the cross-scale ID matching model, so as to update the small target detection model and the cross-scale ID matching model;

[0039] N, K, and M are positive integers, k is a positive integer less than or equal to K, and m is a positive integer less than or equal to M.

[0040] For the above steps S1 and S2, the gun camera video and the dome camera video are videos obtained by shooting the gun camera and the dome camera based on the same target, and the gun camera video and the dome camera video both include M frames; the mth frame of the gun camera video represents any frame of video image in the M frames of video, and the mth frame of the dome camera video represents the same any frame of video image in the M frames of video.

[0041] For the above-mentioned step S3, in the embodiment of the present invention, in the m-th frame of the gun camera video and the m-th frame of the ball camera video, the number of the overview target frames and the number of the enlarged target frames may be inconsistent, so that enlarged target frames that fail to match will appear in the preliminary matching of step S3. For the enlarged target frames that fail to match the preliminary matching, the embodiment of the present invention further expands the matching pool for supplementary matching, that is, the gun camera video is no longer limited to the video image of the m-th frame.

[0042] Based on this, the detailed implementation of the above step S3 includes:

[0043] S31, if the k-th enlarged target frame has the same ID as any one of the N overview target frames, the k-th enlarged target frame and the overview target frame with the same ID are preliminarily matched successfully, and the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame are the overview target frame and the enlarged target frame having a corresponding relationship;

[0044] S32. If the k-th enlarged target frame does not have the same ID as any of the N panoramic target frames, the preliminary matching of the k-th enlarged target frame with the N panoramic target frames fails, and the k-th enlarged target frame is supplementarily matched with all the panoramic target frames of the previous m frames of the gun camera video; wherein, the k-th enlarged target frame successfully obtained by the supplementary matching and the panoramic target frame with the same ID as the k-th enlarged target frame in the previous m frames of the gun camera video are the panoramic target frame and enlarged target frame with a corresponding relationship; the ID data of the k-th enlarged target frame obtained by the supplementary matching failure is the missed target data of the gun camera.

[0045] After preliminary matching and supplementary matching, the embodiment of the present invention obtains the k-th enlarged target frame that succeeds in preliminary matching, the panoramic target frame with the same ID as the k-th enlarged target frame, and the k-th enlarged target frame that succeeds in supplementary matching, and the panoramic target frame with the same ID as the k-th enlarged target frame in the first m frames of the gun camera video as the panoramic target frames with corresponding relationships; and also obtains the ID data of the k-th enlarged target frame that fails in preliminary matching and supplementary matching, as the missed target data of the gun camera. Therefore, it can be understood that the "results of preliminary matching and supplementary matching" in step S4 are the panoramic target frames with corresponding relationships obtained through the above steps S31 and S32, and the missed target data of the gun camera.

[0046] It should be noted that the ID in the above steps S31 and S32 represents coordinates. For example, the ID of the k-th magnified target frame is based on the coordinates of vertex A and vertex B of the k-th magnified target frame, and vertex A and vertex B are diagonally opposite. Similarly, the ID of any overview target frame is based on the coordinates of vertex A' and vertex B' of the overview target frame, and vertex A' and vertex B' are diagonally opposite.

[0047] In the above step S32, the judgment method of supplementary matching is similar to that of preliminary matching, and illustratively includes:

[0048] If the first m frames of gun camera video include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame in the first m frames of gun camera video are successfully supplemented and matched; if the first m frames of gun camera video do not include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame is supplemented and matched unsuccessfully, and the ID data of the k-th enlarged target frame is obtained.

[0049] In addition, for the above step S3, since the ball camera video is an enlargement of a smaller part in the gun camera video, for a target image point, the image point coordinates in the ball camera video and the image point coordinates in the gun camera video are different, but there is a mapping relationship between the two, and usually this mapping relationship can be represented by a linear function. In the embodiment of the present invention, in order to ensure accuracy, for each frame of the ball camera video and the gun camera video, the linear function is recalculated and the mapping relationship is reestablished. Therefore, before the above step S3, it also includes:

[0050] Calculate the image mapping relationship between the mth frame of the gun camera video and the mth frame of the ball camera video, and transform the image point coordinates of the mth frame of the ball camera video;

[0051] The IDs of the K enlarged target frames are represented by the converted image point coordinates of the m-th frame of the ball camera video, and the IDs of the N panoramic target frames are represented by the image point coordinates of the m-th frame of the gun camera video.

[0052] The embodiment of the present invention does not limit the specific process of the linear function representing the image mapping relationship and the conversion of the image point coordinates. In one implementation, it is assumed that the linear function representing the image mapping relationship is: x_q = f_x*x_b+a; y_q = f_y*y_b+b; where (x_q, y_q) is a point in the image of the gun camera video, (x_b, y_b) is a point in the image of the ball camera video, and f_x, f_y, a, and b are linear parameters that need to be calculated. At this time, it is only necessary to know that there is a set of target frames (xq1, yq1), (xq2, yq2) and (xb1, yb1), (xb2, yb2) with matching IDs in the previous frame video or the previous n frame queue (without moving the ball camera), and the corresponding fx, fy, a, and b values ​​can be calculated, that is, the coordinate mapping relationship between the image point of the current frame gun camera video and the image point of the ball camera video.

[0053] In one embodiment, the image point coordinates of the m-th frame gun camera video may also be converted, and then the IDs of the N panoramic target frames are represented by the converted image point coordinates of the m-th frame gun camera video, and the IDs of the K enlarged target frames are represented by the image point coordinates of the m-th frame dome camera video. Finally, for the enlarged target frames that are not matched by the cross-scale ID matching model, the corresponding panoramic target frames can be preliminarily matched and supplementarily matched with the original enlarged target frames based on the derived calculation, that is, the above-mentioned preliminary matching and supplementary matching are performed based on the IDs of the N panoramic target frames represented by the converted image point coordinates of the m-th frame gun camera video and the IDs of the K enlarged target frames of the m-th frame dome camera video.

[0054] For the above step S5, the first m frame queue is used to store the first m frame information. Exemplarily, in the first m frame queue, the information stored in each frame queue includes: a global unified ID, an overview target frame ID, an enlarged target frame ID, a gun camera video image, and a ball camera video image. As shown in Table 1 below, an embodiment of the present invention also provides the information stored in each frame queue and its organization method. In addition, as shown in the third row of Table 1 below, if a frame only includes an enlarged target frame and there is no matching full library template library, then in the information stored in this frame queue, the global unified ID and the overview target frame ID are empty.

[0055] Global unified ID Overview target frame ID Zoom in on the target frame ID Bolt video image Ball machine video image idx [x1,y1,x2,y2] [x1,y1,x2,y2] Mat Mat None None [x1,y1,x2,y2] None Mat

[0056] For the above step S5, in a preferred implementation, when the training data set of the small target detection model is greater than the first preset number of images, the small target detection model is updated;

[0057] When the training data set of the cross-scale ID matching model is greater than a second preset number of images, the cross-scale ID matching model is updated.

[0058] By setting the image quantity threshold, the small target detection model and cross-scale ID matching model are automatically updated.

[0059] like Figure 2 As shown, the embodiment of the present invention further provides an optimization device 20 based on the gun-ball linkage monitoring system, comprising:

[0060] The overview target frame acquisition module 21 is used to acquire N overview target frames, where the N overview target frames are acquired by taking M frames of gun camera video and processing each frame of gun camera video through a small target detection model;

[0061] The enlarged target frame acquisition module 22 is used to obtain K enlarged target frames, where the K enlarged target frames are obtained by processing M frames of dome camera videos shot by the dome camera and each frame of the dome camera video through the dome camera detection model; wherein the gun camera video and the dome camera video are videos shot by the gun camera and the dome camera based on the same target;

[0062] The preliminary matching and supplementary matching module 23 is used to perform preliminary matching and supplementary matching on the kth magnified target frame and the Nth panoramic target frames through a cross-scale ID matching model for the mth frame of the gun camera video and the mth frame of the ball camera video;

[0063] The result data acquisition module 24 is used to obtain the full view target frame and the enlarged target frame with corresponding relationships and the missed target data of the gun camera according to the results of the preliminary matching and the supplementary matching;

[0064] The algorithm optimization module 25 is used to store the corresponding panoramic target frame and the enlarged target frame into the first m frame queue, and store the missed target data of the gun camera into the training data set of the small target detection model and the training data set of the cross-scale ID matching model to update the small target detection model and the cross-scale ID matching model;

[0065] N, K, and M are positive integers, k is a positive integer less than or equal to K, and m is a positive integer less than or equal to M.

[0066] Therefore, the optimization device based on the gun-ball linkage monitoring system provided in the embodiment of the present invention can execute the optimization method based on the gun-ball linkage monitoring system provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0067] Figure 3 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0068] like Figure 3 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0069] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0070] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 31 executes the various methods and processes described above, such as the optimization method based on the gun-ball linkage monitoring system.

[0071] In some embodiments, the data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0072] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0073] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0074] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0075] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0076] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0077] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0078] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0079] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An optimization method based on a gun-ball linkage monitoring system, characterized in that: include: Obtain N panoramic target frames, where the N panoramic target frames are obtained by taking M frames of gun camera video and processing each frame of the gun camera video through a small target detection model; Obtain K magnified target frames, where the K magnified target frames are obtained by taking M frames of dome camera videos and processing each frame of dome camera videos through a dome camera detection model; wherein the gun camera video and the dome camera video are videos taken by the gun camera and the dome camera based on the same target; For the mth frame of the gun camera video and the mth frame of the ball camera video, the kth magnified target frame and the N full-view target frames are preliminarily matched and supplemented through the cross-scale ID matching model; According to the results of the preliminary matching and the supplementary matching, the overall target frame and the enlarged target frame with corresponding relationships and the missed target data of the gun camera are obtained; The corresponding overview target frame and the enlarged target frame are stored in the first m frame queue, and the missed target data of the gun camera are stored in the training data set of the small target detection model and the training data set of the cross-scale ID matching model to update the small target detection model and the cross-scale ID matching model; N, K, and M are positive integers, k is a positive integer less than or equal to K, and m is a positive integer less than or equal to M.

2. The optimization method based on the gun-ball linkage monitoring system as claimed in claim 1, characterized in that: For the mth frame of the gun camera video and the mth frame of the ball camera video, the kth magnified target frame and the N full-view target frames are preliminarily matched and supplemented through the cross-scale ID matching model, including: If the k-th enlarged target frame has the same ID as any one of the N overview target frames, the k-th enlarged target frame and the overview target frame with the same ID are preliminarily matched successfully, and the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame are the overview target frame and enlarged target frame with a corresponding relationship; If the k-th enlarged target frame does not have the same ID as any of the N panoramic target frames, the preliminary matching of the k-th enlarged target frame and the N panoramic target frames fails, and the k-th enlarged target frame is supplementarily matched with all the panoramic target frames of the previous m frames of the gun camera video; wherein, the k-th enlarged target frame successfully obtained by the supplementary matching and the panoramic target frame with the same ID as the k-th enlarged target frame in the previous m frames of the gun camera video are the panoramic target frame and enlarged target frame with a corresponding relationship; the ID data of the k-th enlarged target frame obtained by the supplementary matching failure is the missed target data of the gun camera.

3. The optimization method based on the gun-ball linkage monitoring system as claimed in claim 2, characterized in that: The k-th enlarged target frame is supplemented and matched with all the panoramic target frames of the previous m frames of the gun camera video, including: If the first m frames of gun camera video include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame and the overview target frame with the same ID as the k-th enlarged target frame in the first m frames of gun camera video are successfully supplemented and matched; if the first m frames of gun camera video do not include an overview target frame with the same ID as the k-th enlarged target frame, then the k-th enlarged target frame is supplemented and matched unsuccessfully, and the ID data of the k-th enlarged target frame is obtained.

4. The optimization method based on the gun-ball linkage monitoring system according to any one of claims 1 to 3, characterized in that: In the first m frame queues, each frame queue stores the following information: global unified ID, overview target frame ID, zoomed target frame ID, gun camera video image, and ball camera video image.

5. The optimization method based on the gun-ball linkage monitoring system as claimed in claim 4, characterized in that: If the mth frame only includes the kth enlarged target frame, the global unified ID and the overview target frame ID are empty.

6. The optimization method based on the gun-ball linkage monitoring system according to any one of claims 1 to 3, characterized in that: When the training data set of the small target detection model is greater than a first preset number of images, updating the small target detection model; When the training data set of the cross-scale ID matching model is greater than a second preset number of images, the cross-scale ID matching model is updated.

7. The optimization method based on the gun-ball linkage monitoring system according to any one of claims 1 to 3, characterized in that: For the mth frame of the gun camera video and the mth frame of the ball camera video, the kth magnified target frame and the N full-view target frames are preliminarily matched and supplemented by the cross-scale ID matching model, including: Calculate the image mapping relationship between the mth frame of the gun camera video and the mth frame of the ball camera video, and transform the image point coordinates of the mth frame of the ball camera video; The IDs of the K enlarged target frames are represented by the converted image point coordinates of the m-th frame of the ball camera video, and the IDs of the N panoramic target frames are represented by the image point coordinates of the m-th frame of the gun camera video.

8. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the optimization method based on the gun-ball linkage monitoring system as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the optimization method based on the gun-ball linkage monitoring system as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the optimization method based on the gun-ball linkage monitoring system as described in any one of claims 1 to 7.

Images