Image transmission system and method for night vision scene
By adopting the RTSP protocol and the method of separating image acquisition and processing in the night vision imaging system, high-quality and low-latency wireless image transmission is achieved, solving the problem of limited application scenarios of night vision imaging systems in the prior art, and improving the Image quality and user operation experience.
Patent Information
- Application Number
- CN202411224700.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-06
AI Technical Summary
The existing true color night vision imaging system has limited the application scenarios of night vision imaging due to its high computing power and sensitivity to image quality, and there is a lack of high-quality and low-delay wireless image transmission products on the market.
A system including image acquisition and streaming equipment and image processing equipment is designed, and image transmission is carried out using RTSP real-time streaming protocol, and wireless image transmission and camera parameter adjustment are realized through image preprocessing module, camera and routing module, to separate images Acquisition and processing allow efficient image processing on back-end devices.
The application scenarios of night vision enhancement system have been broadened, high-quality night vision image transmission is provided, delay is reduced, and the convenience of user operation is improved through a simple UI interface and a variety of parameter adjustment modes.
Smart Images

Figure CN119094703A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of night vision imaging, and in particular to a night vision imaging system and method based on wireless transmission and wireless parameter adjustment. Background Art
[0002] In recent years, due to the rapid development of artificial intelligence and the rapid increase in computer computing power, night vision enhancement through neural networks has attracted widespread attention. However, in the current true color night vision imaging system, due to the high computing power requirements and sensitivity to image quality, most cameras in the system need to be connected to the computing device by wire, which limits the application scenarios of night vision imaging.
[0003] With the rapid development of wireless communication technology, especially WIFI technology, wireless image transmission technology has entered the human vision. At present, more and more manufacturers in the field of surveillance are pursuing wireless cameras. However, in the field of true color wireless imaging, there is no good product on the market that can achieve high image quality and low latency. Summary of the invention
[0004] In order to broaden the application scenarios of true color night vision imaging systems and realize wireless image transmission technology with high image quality and low latency, the present invention proposes an image transmission system and method for night vision scenes.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] An image transmission system for night vision scenes, the image transmission system comprising: an image acquisition and streaming device, used to acquire image data in darker scenes, pre-process the acquired images, encode the image data, and stream the encoded images; an image processing device, used to stream the images streamed by the image acquisition and streaming device, decode the streamed data, pre-process the decoded data, perform night vision enhancement on the pre-processed data, and display the night vision enhanced image data.
[0007] Furthermore, the image acquisition and streaming device includes an image preprocessing module, a camera, a camera power line and an image preprocessing module power line; the image preprocessing module is connected to the camera.
[0008] Furthermore, the image preprocessing module includes a preprocessing module, an image encoding module and a data streaming module, which are respectively used for preprocessing the acquired image, encoding the image data and streaming the encoded image.
[0009] Furthermore, the image processing device includes a routing module and an image streaming processing module. The routing module is used to provide communication between the image acquisition and streaming device and the image streaming processing module. The routing module must be able to forward the information of the image streaming processing module and the image preprocessing module to the corresponding address; the image streaming processing module includes a data stream pulling module, an image enhancement module and an image display module, which are respectively used to pull and process the image data stream, complete night vision enhancement and display the night vision enhanced image or video.
[0010] The present invention also provides a method for utilizing the above-mentioned image transmission system for night vision scenes, comprising the following steps: the image acquisition and streaming device acquires image data in a darker scene, and then pre-processes the acquired image data, encodes the pre-processed image data and streams the encoded image to an image processing device; the image processing device streams the image data pushed by the image acquisition and streaming device, and then decodes the pulled image data, pre-processes the decoded data, performs night vision enhancement, and displays the night vision enhanced image data.
[0011] Furthermore, the streaming process of the image acquisition and streaming device and the streaming process of the image processing device use the RTSP real-time streaming protocol.
[0012] Furthermore, the image acquisition and streaming device includes an image preprocessing module, a camera, a camera power cord and an image preprocessing module power cord. The image preprocessing module automatically adjusts the camera parameters and then preprocesses the image data collected by the camera. The preprocessed image data is encoded with H264 and encapsulated into RTSP format data, and then the RTSP format data is streamed to an address accessible to the image streaming processing module.
[0013] Furthermore, the image processing device includes a routing module and an image stream pulling processing module, and the specific processing steps are as follows:
[0014] 1) Applying for stream pulling from an RTSP server based on the TCP protocol, wherein the RTSP server is a corresponding port of the image preprocessing module, and the stream pulling operation is pulling the RTSP video stream;
[0015] 2) Unpack the pulled RTSP video stream into an H264 data stream, and decode the H264 data stream into a YUV422 pixel format image;
[0016] 3) Convert the pixel format of the image obtained in step 2) to obtain an RGB24 format image;
[0017] 4) The image obtained in step 3) is enhanced using a night vision enhancement algorithm;
[0018] 5) Send the enhanced image to the UI interface for display.
[0019] Furthermore, the UI interface is used to display night vision enhanced images and wirelessly adjust camera parameters; the wireless adjustment of camera parameters includes automatic adjustment mode and manual adjustment mode.
[0020] Furthermore, the operation steps of the manual adjustment mode include:
[0021] a) Adjust the camera's white balance, exposure time, exposure gain, and noise reduction algorithm control parameters through the UI interface;
[0022] b) encoding the parameter adjustment information and transmitting it to the image acquisition and streaming device;
[0023] c) The control command is forwarded to the fixed port of the image acquisition and streaming device through the routing module;
[0024] d) The image acquisition and streaming device port receives and decodes the instruction;
[0025] e) adjusting parameters of the camera according to the decoded content.
[0026] Compared with the prior art, the present invention has the following beneficial effects:
[0027] (1) In the system of the present invention, night vision processing can be separated from camera acquisition images, thereby broadening the application scenarios of the night vision enhancement system;
[0028] (2) Compared with the night vision monitoring currently on the market, the present invention can use a better night vision processing program for image processing on a device with stronger back-end computing power due to the separation of image acquisition and night vision processing;
[0029] (3) By using the RTSP protocol for low-latency data transmission, it provides better night vision quality with a latency no higher than that of mainstream night vision surveillance on the market;
[0030] (4) The present invention also provides a concise UI interface and a variety of adjustable parameters, making it more convenient for users to operate. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a schematic diagram of the system structure of the present invention;
[0032] Figure 2 This is a schematic diagram of the working process of the image acquisition and streaming device of the present invention;
[0033] Figure 3 It is a schematic diagram of the working process of the image processing device of the present invention;
[0034] Figure 4It is a schematic diagram of the wireless parameter adjustment process of the present invention. DETAILED DESCRIPTION
[0035] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0036] The present invention provides multiple implementation schemes. The various optional methods introduced in the implementation schemes can be combined and cross-referenced with each other without conflict, thereby extending multiple possible implementation schemes.
[0037] Reference Figure 1 , an image transmission system for night vision scenes, mainly including an image acquisition and streaming device and an image processing device. Among them, the image acquisition and streaming device is used to acquire image data in dark scenes, pre-process the acquired images, encode the image data, and stream the encoded images; the image processing device is used to pull the stream images pushed by the image acquisition and streaming device, decode the pulled stream data, pre-process the decoded data, input the pre-processed data into a night vision enhancement model, and display the night vision enhanced image data.
[0038] The image acquisition and streaming device includes an image preprocessing module, a camera, a camera power cord, and an image preprocessing module power cord. The camera is used to complete the acquisition of image data in darker scenes. The image preprocessing module includes a preprocessing module, an image encoding module, and a data streaming module, which are respectively used to preprocess the acquired image, encode the image data, and stream the encoded image, wherein the preprocessing includes image brightness analysis and image pixel format conversion using hardware acceleration; the camera power cord and the image preprocessing module power cord are respectively used to power the camera and the image preprocessing module.
[0039] The image processing device includes a routing module and an image streaming processing module. The routing module is used to provide communication between the image acquisition and streaming device and the image streaming processing module. The routing module needs to be able to forward the information of the image streaming processing module and the image preprocessing module to the corresponding address, so as to facilitate the wireless image transmission and wireless camera parameter adjustment between the image acquisition and streaming device and the image processing device. The image streaming processing module includes a data stream pulling module, an image enhancement module and an image display module, which are respectively used to pull and process the image data stream, complete night vision enhancement and display the night vision enhanced image or video. The streaming and pulling process of this embodiment uses the RTSP real-time streaming transmission protocol, which has lower latency and better scalability than other protocols commonly used for image transmission. The image acquisition and streaming device communicates with the image processing device through the routing module. A specific data stream push address is set in the image streaming module. If the image processing device can access the corresponding address through the routing module, the data stream can be transmitted to the local of the image processing device through network transmission.
[0040] As a preferred solution of this example, the image preprocessing module includes a central processing unit, an encoding device and a network card, which can use but is not limited to RK3588. The image streaming processing module can use but is not limited to a PC host with an Nvdia graphics card. The PC host should have an image display module, and a computer display screen can be selected. The routing module can select a router. Preferably, the router can select a gigabit router. The camera power supply is a 12V3A round hole power cord connection, and the image preprocessing module power supply is a 12V2A power cord.
[0041] Please refer to Figure 1 As shown, a USB cable is used to connect the image preprocessing module in the image acquisition and streaming device with the camera, and a Category 6 network cable is used to connect the PC with the router or the PC is connected to the WIFI of the router.
[0042] When the image transmission system of this embodiment is used in a night vision scene, it is necessary to first turn on the power of the camera, the power of the image preprocessing module, the power of the router, and the PC. The special thing is that the image preprocessing module needs to detect whether the camera is started when it is started. If it is detected that the camera is not started, the preprocessing program will not be started. Therefore, the camera power must be turned on before the image preprocessing module power.
[0043] The image preprocessing module will automatically start the program after it is powered on, and the image acquisition and streaming device has started Figure 2The workflow of the image acquisition and streaming device. After all modules are turned on, wait for the router to be initialized. When the PC can recognize the router signal, you can open the router management interface to check whether the image preprocessing module is connected to the router LAN. The image preprocessing module will automatically connect to the router LAN. The image preprocessing module automatically adjusts the camera parameters, preprocesses the camera captured images, encodes the preprocessed images and encapsulates them into format data, and streams the format data to the address accessible to the image streaming processing module.
[0044] In the workflow of the image acquisition and streaming device, the image format acquired by the camera needs to be converted to an image pixel format before it can be used for image encoding. The pixel format conversion is from RGB format to YUYV format. The image encoding is to encode the data converted into YUYV format into H264 data. The image transmission requires encapsulation of H264 data. The encoded H264 data is encapsulated into an RTSP format data stream through the FFmpeg audio and video processing library, so that a lower bit rate can be obtained while ensuring image quality. The RTSP format data stream is also transmitted to the fixed port of the image preprocessing module in the image acquisition and streaming device through the FFmpeg audio and video processing library, and communicates with the router through the network card to ensure that the image streaming processing module can obtain the RTSP format data stream. In order to ensure the stability of data transmission, the transmission protocol of RTSP data is defined as the TCP protocol in this embodiment.
[0045] After the image preprocessing module is connected to the router LAN, you can open the PC program and start Figure 3 The workflow of the image processing device. When the PC program is started, the process of pulling and processing the stream is started. The process of pushing the stream to pulling the stream is as follows: the image acquisition and pushing stream device pushes the encoded data to a fixed port of the image preprocessing module. The image preprocessing module is connected to the router WIFI and is in the same local area network as the PC. The PC accesses the fixed port of the image preprocessing module through the local area network to pull the RTSP data stream.
[0046] After the PC program is started, a UI screen written in QT5 will appear, and the UI screen includes an image display module and a parameter adjustment module. In order to ensure the simplicity of the screen, the UI display screen separates the image display module from the parameter adjustment module. When the user is satisfied with the picture quality, the parameter adjustment module can be closed, leaving only the image display module for display; at the same time, the parameter adjustment module compresses the types of control parameters and only retains the adjustment parameters that have a great impact on the picture quality; in addition, the parameter adjustment module integrates control parameter methods with similar functions, such as the exposure gain parameter, which integrates the camera gain and the image processing algorithm gain, which simplifies user operation and reduces learning costs. The image display module is used to display the original or night vision enhanced picture, and the module window can be resized. The parameter adjustment module has two modes: automatic parameter adjustment and manual parameter adjustment, which are controlled by the Auto / Matual button. When the parameter adjustment module automatically adjusts the parameters, it can automatically adjust the picture white balance, exposure time, exposure gain parameters and noise reduction algorithm control parameters to achieve a picture suitable for human eyes to watch; in addition to manually adjusting the parameters included in the automatic adjustment, the parameter adjustment module can also adjust parameters such as picture brightness, sharpness and transparency.
[0047] The process of automatically adjusting parameters is as follows: 1) The image preprocessing module adjusts the parameters by processing the camera acquisition parameters. 2) After the PC starts the program, the parameter adjustment module will periodically obtain the current camera-related parameters and update the display of the parameter adjustment module UI. The process of obtaining the current camera-related parameters can be found in Figure 4 As described above, the execution status of a specific instruction is returned as the value of the required parameter.
[0048] The process of manually adjusting parameters can be referred to Figure 4 The specific process is as follows: 1) adjust the required parameters in the interface of the parameter adjustment module. 2) click the set button to start transmitting the parameter adjustment instruction. 3) the PC and the image preprocessing module both use the UDP protocol to send and receive parameter instructions. 4) the image preprocessing module processes the received relevant instructions and makes corresponding camera parameter change operations. 5) after completing the operation, it will return OK to indicate that the transmission parameters have been successfully modified.
[0049] The specific process of the manual parameter adjustment process 3) is as follows: 1) When the image preprocessing module program and the PC program are started, a fixed port will be bound as the port for UDP transmission. 2) When the PC needs to transmit relevant parameters, relevant instructions are sent to the fixed port of the image preprocessing module through the local area network. 3) When the image preprocessing module returns to the execution state, it will return a status message according to the address when the PC sent the instruction.
[0050] When the image transmission system needs to be shut down, the stop button of the parameter adjustment module of the UI needs to be clicked on the PC. According to actual needs, the router and the image acquisition and streaming device are generally in a normally open state, and no special opening and closing means are required. If necessary, the power can be directly turned off.
Claims
1. An image transmission system for night vision scenes, characterized in that: The image transmission system comprises: Image acquisition and streaming equipment, used to acquire image data in dark scenes, pre-process acquired images, encode image data, and stream encoded images; The image processing device is used to pull the image pushed by the image acquisition and streaming device, decode the pulled stream data, pre-process the decoded data, perform night vision enhancement on the pre-processed data and display the night vision enhanced image data.
2. The image transmission system for night vision scenes according to claim 1, characterized in that: The image acquisition and streaming device comprises an image preprocessing module, a camera, a camera power line and an image preprocessing module power line; the image preprocessing module is connected to the camera.
3. The image transmission system for night vision scenes according to claim 1, characterized in that: The image preprocessing module includes a preprocessing module, an image encoding module and a data streaming module, which are respectively used for preprocessing the captured image, encoding the image data and streaming the encoded image.
4. The image transmission system for night vision scenes according to claim 1, characterized in that: The image processing device includes a routing module and an image streaming processing module. The routing module is used to provide communication between the image acquisition and streaming device and the image streaming processing module. The routing module needs to be able to forward the information of the image streaming processing module and the image preprocessing module to the corresponding address; the image streaming processing module includes a data stream pulling module, an image enhancement module and an image display module, which are respectively used to pull and process the image data stream, complete night vision enhancement and display the night vision enhanced image or video.
5. A method for utilizing an image transmission system for night vision scenes as claimed in claim 1, characterized in that: The steps include: The image acquisition and streaming device acquires image data in a darker scene, then preprocesses the acquired image data, encodes the preprocessed image data, and streams the encoded image to the image processing device; The image processing device pulls the image data pushed by the image acquisition and streaming device, then decodes the pulled image data, pre-processes the decoded data, performs night vision enhancement, and displays the image data after night vision enhancement.
6. The method according to claim 5, characterized in that The streaming process of the image acquisition and streaming device and the streaming process of the image processing device use the RTSP real-time streaming protocol.
7. The method according to claim 6, characterized in that The image acquisition and streaming device includes an image preprocessing module, a camera, a camera power cord and an image preprocessing module power cord. The image preprocessing module automatically adjusts the camera parameters, and then preprocesses the image data collected by the camera. The preprocessed image data is encoded with H264 and encapsulated into RTSP format data, and then the RTSP format data is streamed to an address accessible to the image streaming processing module.
8. The method according to claim 6, characterized in that The image processing device includes a routing module and an image stream processing module, and the specific processing steps are as follows: 1) Applying for stream pulling from an RTSP server based on the TCP protocol, wherein the RTSP server is a corresponding port of the image preprocessing module, and the stream pulling operation is pulling the RTSP video stream; 2) Unpack the pulled RTSP video stream into an H264 data stream, and decode the H264 data stream into a YUV422 pixel format image; 3) Convert the pixel format of the image obtained in step 2) to obtain an RGB24 format image; 4) The image obtained in step 3) is enhanced using a night vision enhancement algorithm; 5) Send the enhanced image to the UI interface for display.
9. The method according to claim 8, characterized in that The UI interface is used to display night vision enhanced images and wirelessly adjust camera parameters; the wireless adjustment of camera parameters includes automatic adjustment mode and manual adjustment mode.
10. The method according to claim 9, characterized in that The operation steps of the manual adjustment mode include: a) Adjust the camera's white balance, exposure time, exposure gain, and noise reduction algorithm control parameters through the UI interface; b) encoding the parameter adjustment information and transmitting it to the image acquisition and streaming device; c) The control command is forwarded to the fixed port of the image acquisition and streaming device through the routing module; d) The image acquisition and streaming device port receives and decodes the instruction; e) adjusting parameters of the camera according to the decoded content.
Citation Information
Patent Citations
Unmanned aerial vehicle-mounted VR video acquisition and transmission device based on 5G network
CN112714281A
Multimedia ultrasonic conference system
CN117978948A
Sky-view image acquisition device, bright-view image generation device, and dark-view system
CN118096547A
Video stream processing method and apparatus, storage medium, and electronic device
WO2024022317A1
