Highway gun camera and ball camera linkage method and device based on edge guidance and adversarial network

Abstract

The present application proposes a method and device for linking highway gun camera and dome camera based on edge guidance and adversarial network, including the following steps: obtaining the dome camera image to be linked and the edge features of the dome camera image to be linked corresponding to the dome camera image to be linked; constructing a generator module, which takes the dome camera image edge features as input during the training process, and adds noise to the dome camera image edge features through multi-step diffusion to obtain the noise image edge information vector, and then performs multi-step reverse denoising on the noise image edge information vector to obtain a high-resolution dome camera image. This scheme uses the gun camera image edge features as a guide in the multi-step reverse denoising process, eliminates the domain difference between the dome camera image and the gun camera image, so that the obtained high-resolution dome camera image can be directly linked with the gun camera image.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a highway gun camera and ball camera linkage method and system based on edge guidance and adversarial network. Background Art

[0002] Highway monitoring technology has been evolving with the rapid development of video processing and communication technology. In intelligent transportation systems, the widespread application of monitoring cameras has become a reality. In intelligent transportation systems, the same vehicle target can be identified through the linkage of multiple cameras to accurately track the vehicle. However, the different shooting effects of different types of cameras result in low accuracy in vehicle tracking. At present, there are mainly two types of cameras on the road, one is a gun camera and the other is a ball camera. The viewing angle range and image quality of the gun camera are fixed, so the accuracy of vehicle tracking between gun cameras is very high. The main vehicle re-identification on the market is to re-identify the images obtained by the gun camera, while the ball camera is different. Since the ball camera can rotate over a large range, the viewing angle range of the image obtained by the ball camera is quite different from that of the gun camera. In addition, due to the complexity of the highway environment, including different weather conditions and lighting changes, the quality of the image obtained by the ball camera may be affected. Therefore, when tracking a vehicle, the vehicle recognition accuracy of the ball camera is often not high.

[0003] In summary, how to perform vehicle recognition and tracking on images of different viewing angles and qualities acquired by gun cameras and ball cameras, so as to ensure the linkage of gun cameras and ball cameras in intelligent transportation systems is a problem that needs to be solved urgently in the existing technology. Summary of the invention

[0004] The embodiment of the present application provides a method and device for linking a highway gun camera and a dome camera based on edge guidance and an adversarial network. A generator architecture is constructed, and noise is first added to the edge of the dome camera image during the training process of the generator architecture. Then, the edge features of the gun camera image are used as a guide to perform multi-step inverse denoising to obtain a high-resolution dome camera image. The generator architecture can restore the edge information in the dome camera image and eliminate the domain difference between the dome camera image and the gun camera image, so that the obtained high-resolution dome camera image can be directly linked with the gun camera image.

[0005] In a first aspect, an embodiment of the present application provides a highway gun camera and ball camera linkage method based on edge guidance and adversarial network, the method comprising:

[0006] Acquire the dome camera image to be linked and the dome camera image edge features corresponding to the dome camera image to be linked, wherein the dome camera image edge features are the contour features and shape detail features of each object in the dome camera image to be linked;

[0007] Constructing a generator module, wherein the generator module is trained by multiple groups of training samples, each group of training samples is composed of edge features of gun camera images and edge features of ball camera images with the same shooting content. In the training process, the edge features of the ball camera images are firstly used as input, noise is added to the edge features of the ball camera images through multi-step diffusion to obtain a noise image edge information vector, and then the edge features of the gun camera images are used as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator is constructed to perform adversarial training with the generator, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image;

[0008] The ball camera image to be linked is input into the generator module to obtain a high-resolution ball camera image to be linked, and the high-resolution ball camera image to be linked is linked with a gun camera image obtained by any gun camera.

[0009] In a second aspect, the embodiment of the present application provides a highway gun camera and ball camera linkage device based on edge guidance and adversarial network, including:

[0010] An acquisition module is used to acquire the image of the ball camera to be linked and the edge features of the image of the ball camera to be linked corresponding to the image of the ball camera to be linked, wherein the edge features of the image of the ball camera to be linked are the contour features and shape detail features of each object in the image of the ball camera to be linked;

[0011] A construction module is used to construct a generator module, wherein the generator module is trained by multiple sets of training samples, each set of training samples is composed of edge features of gun camera images and edge features of ball camera images with the same shooting content. During the training process, the edge features of the ball camera images are first used as input, noise is added to the edge features of the ball camera images through multi-step diffusion to obtain a noise image edge information vector, and then the edge features of the gun camera images are used as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator is constructed to perform adversarial training with the generator, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image;

[0012] The linkage module is used to input the ball camera image to be linked into the generator module to obtain the high-resolution ball camera image to be linked, and to link the high-resolution ball camera image to be linked with the gun camera image obtained by any gun camera.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a highway gun camera and ball camera linkage method based on edge guidance and adversarial network.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a highway gun camera and ball camera linkage method based on edge guidance and adversarial network.

[0015] The main contributions and innovations of the present invention are as follows:

[0016] The embodiment of the present application uses a gun-and-ball machine to obtain multiple groups of gun camera images and ball camera images with the same content to train the generator architecture. The generator architecture first adds noise to the edge features of the ball camera image through multi-step diffusion, and then uses the edge features of the gun camera image as a guide to obtain the high-resolution ball camera image through multi-step reverse denoising. Since the edge features of the gun camera image are used as a guide in the multi-step reverse denoising process, the edge information in the obtained high-resolution ball camera image is closer to the gun camera image, so it can be directly linked with any gun camera image. During the training process, this solution constructs a discriminator to perform adversarial training with the generator architecture, so that the high-resolution ball camera images generated by the generator architecture are more realistic and more in line with the actual captured ball camera images, further improving the accuracy of the linkage with the gun camera images.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a flow chart of a highway gun camera and ball camera linkage method based on edge guidance and adversarial network according to an embodiment of the present application;

[0020] Figure 2 It is a structural schematic diagram of a highway gun camera and ball camera linkage method based on edge guidance and adversarial network according to an embodiment of the present application;

[0021] Figure 3 It is a structural block diagram of a highway gun camera and ball camera linkage device based on edge guidance and adversarial network according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] Embodiment 1

[0026] The embodiment of the present application provides a method for linking a highway gun camera and a dome camera based on edge guidance and an adversarial network, constructs a generator architecture, and in the training process of the generator architecture, first adds noise to the edge of the dome camera image, and then uses the edge features of the gun camera image as a guide to perform multi-step reverse denoising to obtain a high-resolution dome camera image, so that the generator architecture can restore the edge information in the dome camera image, and eliminate the domain difference between the dome camera image and the gun camera image, so that the obtained high-resolution dome camera image can be directly linked with the gun camera image. Specifically, refer to Figure 1 as well as Figure 2 , the method comprising:

[0027] Acquire the dome camera image to be linked and the dome camera image edge features corresponding to the dome camera image to be linked, wherein the dome camera image edge features are the contour features and shape detail features of each object in the dome camera image to be linked;

[0028] Constructing a generator module, wherein the generator module is trained by multiple groups of training samples, each group of training samples is composed of edge features of gun camera images and edge features of ball camera images with the same shooting content. In the training process, the edge features of the ball camera images are firstly used as input, noise is added to the edge features of the ball camera images through multi-step diffusion to obtain a noise image edge information vector, and then the edge features of the gun camera images are used as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator is constructed to perform adversarial training with the generator, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image;

[0029] The ball camera image to be linked is input into the generator module to obtain a high-resolution ball camera image to be linked, and the high-resolution ball camera image to be linked is linked with a gun camera image obtained by any gun camera.

[0030] In this scheme, Transformer is first used to extract the features of the dome camera image to be linked to obtain the feature map of the dome camera to be linked, and then a fully connected layer is used to project the feature map of the dome camera to be linked to one-dimensional space to obtain the one-dimensional feature matrix of the dome camera image to be linked. Finally, Gaussian filtering is used to extract the edge information in the one-dimensional feature matrix of the dome camera image to be linked to obtain the edge features of the dome camera image to be linked.

[0031] Similarly, training samples are also obtained in the same way. The gun-ball machine is used to obtain gun images and ball camera images with the same content in scenes such as main lines, interchanges, bridges and tunnels on highways. The gun-ball machine can ensure that the collected gun images and ball camera images have the same content, which is helpful for the training of the subsequent generator architecture of this scheme. After that, the transformer is used to extract the features of each gun image and each ball camera image to obtain a gun feature map and a ball camera feature map. The fully connected layer is then used to project the gun feature map and the ball camera feature map into one-dimensional space to obtain a one-dimensional feature matrix of the gun image and a one-dimensional feature matrix of the ball camera image. Finally, Gaussian filtering is used to extract the edge information in the one-dimensional feature matrix of the gun image and the one-dimensional feature matrix of the ball camera image to obtain the edge features of the gun image and the edge features of the ball camera image.

[0032] Specifically, transformer is a network based on the attention mechanism, which can efficiently extract feature representations from images. Especially for surveillance images with time-series relationships, the use of transformer can better understand the complex structure of the image through the attention mechanism. Projecting the gun camera feature map / ball camera feature map into one-dimensional space through the fully connected layer can also greatly reduce the dimension of the data and achieve feature compression, thereby facilitating the subsequent identification of edge information in the gun camera feature map / ball camera feature map through Gaussian filtering. The specific formula is as follows:

[0033]

[0034]

[0035] Among them, FC represents the fully connected layer, Indicates the characteristics of the gun camera / ball camera. is the one-dimensional feature matrix of the gun camera image / the one-dimensional feature matrix of the ball camera image, H represents Gaussian filtering, It is the edge feature of the gun camera image / the edge feature of the ball camera image.

[0036] In some specific embodiments, Gaussian filtering can be used to detect situations where pixel intensity in a gun camera image / ball camera image changes significantly, thereby accurately obtaining edge information. The contour feature extracted in this solution refers to the outer boundary line of a closed area in the gun camera image / ball camera image, and the shape detail feature refers to a specific form inside or on the edge of an object, including but not limited to corners, concave and convex parts, texture changes, etc. For example, the shape details of a car may include the round shape of the headlights, the position of the door handles, the shape of the windows, etc. These details are very helpful in distinguishing different types of vehicles.

[0037] For example, suppose we have a gun-mounted surveillance image that contains a car parked on the side of the road. Through the edge features of the gun-mounted image in the image, we can obtain the contour features of the car, that is, the outer contour lines of the car, which can help us roughly understand the position and posture of the vehicle. Further, by analyzing the edges, we can extract shape detail features, such as the circular outline of the headlights, the rectangular frame of the license plate, etc. These shape details not only help to confirm that this is a vehicle, but also can further distinguish whether it is a car or SUV and other different models, so as to effectively track and re-identify the vehicle.

[0038] In this scheme, the formula for adding noise to the edge features of the ball camera image by multi-step diffusion to obtain the edge information vector of the noise image is expressed as follows:

[0039] ;

[0040] Where T represents the total number of diffusion steps; Indicates the edge features of the ball camera image; represents the edge features of the ball camera image at step t, It is obtained by adding noise in step t-1; Indicates the edge features of the ball camera image at step t-1; represents the noise covariance at step t; I represents the identity matrix, which is used to ensure that the noise added at each step is independent and identical in all dimensions; Used to indicate the mean , the variance is The multivariate Gaussian distribution of , and the noise added at each step follows , The standard deviation of the noise added at each step is used to control the scale of the noise added at each step. Represents the edge features of the ball camera image Diffusion of T steps into the edge information vector of the noise image process, Indicates that at step t, Convert to process.

[0041] Specifically, by gradually adding noise, the generator architecture can better learn the edge information in the edge features of the dome camera image, so in practical applications, the generator can recognize and retain this information even under extreme conditions.

[0042] In this scheme, the formula for obtaining a high-resolution ball camera image by performing multi-step inverse denoising on the edge information vector of the noise image is as follows:

[0043] ;

[0044] in, is the edge feature of the gun image; Indicates that the edge features of the gun image are used as a guide. Reverse denoising to obtain the edge features of the ball camera image in the previous step process; Represents the mean function at step t, which is used to predict the edge features of the ball camera image at step t. represents the variance of the tth step, which is used to control the uncertainty in the inverse denoising process. T-1 is the total number of steps in the inverse denoising process. I represents the unit matrix, which is used to ensure that the noise eliminated in each step is independent and identical in all dimensions.

[0045] Specifically, in the process of multi-step inverse denoising, the original clear edge information can be reconstructed from the noise, so that the generator can restore the edge details of the dome camera image in various complex situations, thereby generating high-resolution dome camera images through continuous iterative training. Moreover, since the edge features of the gun camera image are used as a guide in the process of multi-step denoising, the domain difference between the dome camera image and the gun camera image is further eliminated, and the generalization ability of the generator architecture is enhanced, so that it can handle various complex inputs.

[0046] In this solution, the discriminator judges the quality and authenticity of each high-resolution dome camera image based on the edge features of the real dome camera image corresponding to each high-resolution image.

[0047] Specifically, the generator architecture and the discriminator form an adversarial network to compete, so that the quality and authenticity of the high-resolution image generated by the generator are the same as the actual ball camera image. Furthermore, the loss function in the adversarial network includes a first loss and a second loss. The first loss is used to ensure the discriminator's recognition accuracy of the edge features of the real ball camera image, and the second loss is used to ensure that the edge information of the high-resolution image generated by the generator architecture can deceive the discriminator. The formula is as follows:

[0048] ;

[0049] in, is the total loss of the adversarial network; Represents the edge features of the real ball camera image; represents the output of the discriminator based on the edge features of the real ball camera image; Use logarithms to increase the nonlinearity of the gradient, avoid gradient explosion, and ensure a more stable optimization process; Represents edge information of high-resolution images generated by the generator architecture; It is used to punish the discriminator for judging the edge information of the generated high-resolution image as the edge features of the real dome camera image; It is used to ensure the accuracy of the discriminator in recognizing the edge features of the real ball camera image. The edge information used to ensure that the generator architecture produces high-resolution images that can fool the discriminator.

[0050] That is to say, in the process of adversarial learning, the discriminator can accurately identify the edge information of the high-resolution image during the learning process and determine whether it is a real dome camera image, while the generator architecture can continuously generate more realistic high-resolution dome camera images to deceive the discriminator during the learning process, thereby generating high-quality, high-resolution dome camera images that are closer to real shooting.

[0051] In addition, after the generator architecture training is completed to obtain the generator module, in actual application, the generator module only needs to perform multi-step reverse denoising on the edge features of the input dome camera image to obtain a high-resolution dome camera image, without adding noise to intervene in the discriminator. Since the generator module is guided by the edge features of the gun camera image during the training process, the edge features of the generated high-resolution dome camera image are more inclined to the edge features of the gun camera image, so it can be easily linked with the gun camera image obtained by any gun camera to complete the tracking of the vehicle.

[0052] Embodiment 2

[0053] Based on the same idea, refer to Figure 3 , the application also proposes a highway gun machine and ball machine linkage device based on edge guidance and adversarial network, including:

[0054] An acquisition module is used to acquire the image of the ball camera to be linked and the edge features of the image of the ball camera to be linked corresponding to the image of the ball camera to be linked, wherein the edge features of the image of the ball camera to be linked are the contour features and shape detail features of each object in the image of the ball camera to be linked;

[0055] A construction module is used to construct a generator module, wherein the generator module is trained by multiple sets of training samples, each set of training samples is composed of edge features of gun camera images and edge features of ball camera images with the same shooting content. During the training process, the edge features of the ball camera images are first used as input, noise is added to the edge features of the ball camera images through multi-step diffusion to obtain a noise image edge information vector, and then the edge features of the gun camera images are used as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator is constructed to perform adversarial training with the generator, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image;

[0056] The linkage module is used to input the ball camera image to be linked into the generator module to obtain the high-resolution ball camera image to be linked, and to link the high-resolution ball camera image to be linked with the gun camera image obtained by any gun camera.

[0057] Embodiment 3

[0058] This embodiment also provides an electronic device, referring to Figure 4 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0059] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0060] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0061] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0062] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the highway gun camera and ball camera linkage methods based on edge guidance and adversarial network in the above embodiments.

[0063] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0064] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0065] The input / output device 408 is used to input or output information. In this embodiment, the input information may be a gun camera image and a ball camera image, etc., and the output information may be a high-resolution ball camera image, etc.

[0066] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0067] Acquire the dome camera image to be linked and the dome camera image edge features corresponding to the dome camera image to be linked, wherein the dome camera image edge features are the contour features and shape detail features of each object in the dome camera image to be linked;

[0068] Constructing a generator module, wherein the generator module is trained by multiple groups of training samples, each group of training samples is composed of edge features of gun camera images and edge features of ball camera images with the same shooting content. In the training process, the edge features of the ball camera images are firstly used as input, noise is added to the edge features of the ball camera images through multi-step diffusion to obtain a noise image edge information vector, and then the edge features of the gun camera images are used as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator is constructed to perform adversarial training with the generator, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image;

[0069] The ball camera image to be linked is input into the generator module to obtain a high-resolution ball camera image to be linked, and the high-resolution ball camera image to be linked is linked with a gun camera image obtained by any gun camera.

[0070] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0071] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0072] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 4 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0073] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A highway gun camera and ball camera linkage method based on edge guidance and adversarial network, characterized in that: The following steps are involved: Acquire the dome camera image to be linked and the dome camera image edge features corresponding to the dome camera image to be linked, wherein the dome camera image edge features are the contour features and shape detail features of each object in the dome camera image to be linked; A generator module is constructed. The generator module is trained by multiple sets of training samples. Each set of training samples consists of edge features of gun camera images and edge features of ball camera images with the same shooting content. In the training process, the edge features of the ball camera images are first used as input, and noise is added to the edge features of the ball camera images through multi-step diffusion to obtain the noise image edge information vector. The formula is as follows: Where T represents the total number of diffusion steps; Indicates the edge features of the ball camera image; represents the edge features of the ball camera image at step t, It is obtained by adding noise in step t-1; Indicates the edge features of the ball camera image at step t-1; represents the noise covariance at the tth step; I represents the identity matrix, which is used to ensure that the noise added at each step is independent and identical in all dimensions; Used to indicate the mean , the variance is The multivariate Gaussian distribution of , and the noise added at each step follows , The standard deviation of the noise added at each step is used to control the scale of the noise added at each step. Represents the edge features of the ball camera image Diffusion of T steps into the edge information vector of the noise image process, Indicates that at step t, Convert to The process is then performed by using the edge features of the gun camera image as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator and a generator are constructed for adversarial training, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image; The ball camera image to be linked is input into the generator module to obtain a high-resolution ball camera image to be linked, and the high-resolution ball camera image to be linked is linked with a gun camera image obtained by any gun camera.

2. According to the method of claim 1, the method is characterized in that: First, Transformer is used to extract the features of the dome camera image to be linked to obtain the feature map of the dome camera to be linked. Then, a fully connected layer is used to project the feature map of the dome camera to be linked to one-dimensional space to obtain the one-dimensional feature matrix of the dome camera image to be linked. Finally, Gaussian filtering is used to extract the edge information in the one-dimensional feature matrix of the dome camera image to be linked to obtain the edge features of the dome camera image to be linked.

3. According to the method of claim 1, the method is characterized in that: The formula for performing multi-step inverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image is as follows: in, is the edge feature of the gun image; Indicates that the edge features of the gun image are used as a guide. Reverse denoising to obtain the edge features of the ball camera image in the previous step process; Represents the mean function at step t, which is used to predict the edge features of the ball camera image at step t. represents the variance of the tth step, which is used to control the uncertainty in the inverse denoising process. T-1 is the total number of steps in the inverse denoising process. I represents the unit matrix, which is used to ensure that the noise eliminated in each step is independent and identical in all dimensions.

4. According to the method of claim 1, the method is characterized in that: The discriminator judges the quality and authenticity of each high-resolution dome camera image based on edge features of the real dome camera image corresponding to each high-resolution image.

5. According to the method of claim 1, the method is characterized in that: The loss function in the adversarial network includes a first loss and a second loss. The first loss is used to ensure the discriminator's recognition accuracy of the edge features of the real dome camera image, and the second loss is used to ensure that the edge information of the high-resolution image generated by the generator architecture can deceive the discriminator.

6. A highway gun and ball machine linkage device based on edge guidance and adversarial network, characterized in that: include: An acquisition module is used to acquire the image of the ball camera to be linked and the edge features of the image of the ball camera to be linked corresponding to the image of the ball camera to be linked, wherein the edge features of the image of the ball camera to be linked are the contour features and shape detail features of each object in the image of the ball camera to be linked; The construction module is used to construct a generator module. The generator module is trained by multiple sets of training samples. Each set of training samples consists of edge features of gun camera images and edge features of ball camera images with the same shooting content. In the training process, the edge features of the ball camera image are first used as input, and noise is added to the edge features of the ball camera image through multi-step diffusion to obtain the noise image edge information vector. The formula is as follows: Where T represents the total number of diffusion steps; Indicates the edge features of the ball camera image; represents the edge features of the ball camera image at step t, It is obtained by adding noise in step t-1; Indicates the edge features of the ball camera image at step t-1; represents the noise covariance at step t; I represents the identity matrix, which is used to ensure that the noise added at each step is independent and identical in all dimensions; Used to indicate the mean , the variance is The multivariate Gaussian distribution of , and the noise added at each step follows , The standard deviation of the noise added at each step is used to control the scale of the noise added at each step. Represents the edge features of the ball camera image Diffusion of T steps into the edge information vector of the noise image process, Indicates that at step t, Convert to The process is then performed by using the edge features of the gun camera image as a guide to perform multi-step reverse denoising on the edge information vector of the noise image to obtain a high-resolution ball camera image, and a discriminator and a generator are constructed for adversarial training, wherein the discriminator discriminates the image quality and authenticity of the high-resolution ball camera image; The linkage module is used to input the ball camera image to be linked into the generator module to obtain the high-resolution ball camera image to be linked, and to link the high-resolution ball camera image to be linked with the gun camera image obtained by any gun camera.

7. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute a highway gun camera and ball camera linkage method based on edge guidance and adversarial network as described in any one of claims 1-5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes a highway gun camera and ball camera linkage method based on edge guidance and adversarial network according to any one of claims 1-5.

Images