Multi-node-based adaptive parameter adjustment image recoding transmission method

Abstract

The invention relates to an image recoding transmission method based on multi-node self-adaptive parameter adjustment, which is mainly applied to image recoding transmission and aims at real-time use scenes with high requirements on network bandwidth, image frame rate and low delay. An original YUV422 image is coded in sequence by using a self-defined node thread group number adjustment factor and an image recoding coefficient factor, then strong time sequence association framing processing is performed after parallel image coding is performed on a node thread group, and forwarding is performed according to the time sequence of an image channel by using a polling method. According to the method, the node thread group number adjusting factor and the image recoding coefficient factor can be adaptively adjusted according to the current computer resource parameter set and the image channel number, so that the image recoding coefficient factor is obtained, and the image is coded into the JPEG to be transmitted; and image set strong time sequence, high frame rate, low delay and continuous transmission can be carried out at the optimal network bandwidth utilization rate and image frame rate.

Description

Technical Field

[0001] This invention belongs to the field of image encoding and transmission processing technology, and relates to a method for image re-encoding and transmission based on multi-node adaptive parameter tuning. Background Technology

[0002] Currently, mainstream video transmission primarily involves transmitting raw captured images as continuous bitstreams after H.264 / H.265 encoding, with the receiving end decoding and rendering the video. While handling H.264 / H.265 encoded video streaming and decoding / rendering performs well in a client / server (C / S) architecture, in a browser / server (B / S) architecture, using RTSP or RTMP to pull and decode H.265 encoded video streams presents a challenge that balances high compression efficiency with complex technical implementation. H.265 (HEVC) significantly reduces the bitrate compared to H.264 while maintaining the same image quality, but this comes with higher demands on decoding computing power and challenges in browser compatibility. H.265 decoding is more complex than H.264, placing a greater burden on the CPU / GPU. Pure software decoding is prone to stuttering at high resolutions. Mainstream browsers do not natively support playback of RTSP / RTMP+H.265 videos. Furthermore, the transmission efficiency of RTSP, TCP, or UDP, coupled with transcoding schemes, introduces additional latency. Simultaneous rendering of a large number of video streams puts significant strain on the browser's memory and CPU. B / S architecture also faces significant challenges when dealing with practical application requirements such as low latency, high image quality, high frame rate, and smooth, continuous video. For example, in certain scenarios, if an existing B / S architecture platform outputs YUV422 images and needs to transmit them to a new B / S platform for further processing, the approach of re-encoding the YUV422 images into H264 / H265 streams before transmission will inevitably face even greater challenges.

[0003] Based on the above, while "encoding raw YUV422 images into JPEG single frames for transmission" is not a mainstream solution for continuous video transmission, it possesses irreplaceable advantages in specific scenarios requiring "non-continuous frame transmission, lightweight applications, and priority given to general compatibility." The core logic is that JPEG's high single-frame compression ratio, extremely low decoding threshold, and cross-platform compatibility perfectly match the core requirement of scenarios that "require only static or intermittent images, not continuous video streams," while avoiding the problems of excessively large raw YUV422 data size and high H264 / H265 encoding complexity. The main application scenarios include, but are not limited to, the following five typical scenarios: transmission of "captured frames" from surveillance or equipment; "lightweight IoT image transmission" with low computing power or low bandwidth; "image preview or archiving" on general platforms; single-frame high-definition data transmission in special industries; and image intelligent recognition scenarios connecting existing B / S platforms to new B / S platforms during data aggregation and information unification. Previously, encoding raw YUV422 images into JPEG single frames only involved setting the quality factor and did not fully utilize system-level comprehensive parameter analysis. Therefore, the existing method of encoding raw YUV422 images into JPEG single frames for transmission mainly suffers from the following problems: (1) The actual compression ratio is affected by the quality factor during JPEG encoding and the complexity of the image content, and is limited by a single factor and cannot be adaptively adjusted; (2) When converting YUV422 to JPEG and then transmitting it, the relationship between concurrency, computer resource parameter set and the number of image set channels was not fully considered; (3) When transmitting multi-channel images concurrently, the system did not take a comprehensive approach and ignored the relationship between computer resources and transmission performance of the entire system. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a method for image recoding and transmission based on multi-node adaptive parameter tuning. This method can obtain the image recoding coefficient factor based on the current computer resource parameter set, the number of image channels, the adaptive adjustment factor of the number of node thread groups and the image recoding coefficient factor, and encode the image into JPEG for retransmission. This method can achieve strong temporal, high frame rate, low latency and continuous transmission of image sets with optimal network bandwidth utilization and image frame rate.

[0005] The technical solution of this invention is: a method for image recoding and transmission based on multi-node adaptive parametric tuning, comprising: The number of node thread groups for the current image channel is obtained by constructing a node thread group number adjustment factor using a custom set of computer resource parameters, the data relationship between the number of image channels and the number of node thread groups; By utilizing the data relationship between a custom node thread group number adjustment factor and the image set frame rate, an image adaptive adjustment factor is constructed, resulting in the image recoding coefficient factor; The original YUV422 images in shared memory are timestamped to form an image precoding dataset, which is then placed into the resource pool. The multi-node thread group extracts the corresponding image precoding dataset from the resource pool according to the strong temporal dataset and marks the node thread group encoding; Each node thread group re-encodes the YUV422 images in the image precoding dataset encoded by the marked node thread group according to the image recoding coefficient factor to form a JPEG image set; Each node thread group performs strong temporal correlation frame assembly processing on the re-encoded JPEG image set to obtain the framed image dataset; After the framed image dataset is sorted by a strong timing polling timing window, it is then forwarded via WebSocket.

[0006] The node thread group number adjustment factor, constructed using a custom set of computer resource parameters and the data relationship between the number of image channels and the number of node thread groups, is used to obtain the current number of node thread groups for the image channel, including: Set the node thread group number adjustment factor : ; in, Indicates standardized network bandwidth. ; This represents the standardized number of computer CPU cores. ; This refers to the standardized computer clock speed. ; This represents standardized memory bandwidth. Represents the standardized number of image channels. ; , , as well as These are all parameters from the computer resource parameter set; express The adjustment factor, and so on; Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was calculated using the least squares method. The initial value; The calculation yielded Initial values ​​include: By minimizing the sum of squared errors The unique analytical expression of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial adjustment factors obtained are as follows: ; Real-time acquisition of independent variable matrix During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; Adjust the factor based on the obtained number of node thread groups. The number of node thread groups was calculated.

[0007] ; Where N represents the number of node thread groups, and the subscript c represents the number of image channels.

[0008] The image adaptive adjustment factor, constructed by utilizing the data relationship between a custom node thread group number adjustment factor and the image set frame rate, yields the image recoding coefficient factor, including: Let the image adaptive adjustment factor be set. for: ; in, The adjustment factor for the number of node thread groups is... , The frame rate of the standardized image set is... ,in, ; express The adjustment factor, and so on; Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was solved using the least squares method, resulting in... The initial value; Real-time acquisition of independent variable matrix During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; Based on image adaptive adjustment factor Image recoding coefficient factor

[0009] .

[0010] The calculation yielded Initial values ​​include: By minimizing the sum of squared errors The unique analytic value of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial factor values ​​obtained are as follows: .

[0011] The node thread group encoding includes the node number and the timestamp number in the image pre-encoding dataset.

[0012] Each node thread group re-encodes the YUV422 images in the image precoding dataset encoded by the marked node thread group to form a JPEG image set based on the image recoding coefficient factor. This includes: each node thread group normalizes the solved image recoding coefficient factor into an image coding quality factor, encodes the YUV422 images into JPEG images, and forms a JPEG image set.

[0013] Each node thread group performs strong temporal association framing processing on the re-encoded JPEG image set to obtain a framed image dataset, including: each node thread group performs temporal association framing according to a custom frame format and the temporal order of the JPEG images in the JPEG image set to obtain a framed image dataset.

[0014] The process of using a strong-time polling timing window to perform sliding window sorting on the framed image dataset before WebSocket forwarding includes: By periodically polling to trigger the updating and calculation of the sliding window, real-time and continuous processing of image datasets is achieved, specifically including: Set polling period Window size Sliding step size ,in, Image frame rate; After performing strong temporal sorting on the image dataset retrieved by the sliding window according to the parameters set above, it is then forwarded via WebSocket.

[0015] The advantages of this invention compared to the prior art are: (1) The method of the present invention can significantly improve the systematic performance of YUV422 encoded JPEG retransmission, and can meet the continuous transmission of each channel image at a frame rate of 30 frames / second or more, with an end-to-end latency of less than 200ms on the B / S architecture platform.

[0016] (2) The method of this invention changes the existing situation of using a relatively single influence factor parameter for encoding and transmission. It can obtain the image recoding coefficient factor based on the current computer resources, the number of image channels, the adaptive adjustment factor of the number of node thread groups and the image recoding coefficient factor, and encode the image into JPEG before transmission. It can achieve strong temporal, high frame rate, low latency and continuous transmission of image sets with optimal network bandwidth utilization and image frame rate.

[0017] (3) The method of the present invention also fully considers the frame order after JPEG encoding, the number of image set channels and the expandable task parameters. After framing according to custom rules, it sends the data through Websocket, realizing the strong temporal attributes and the synchronization of task attributes of the receiving end when multiple channel image datasets are concurrent. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method for image recoding and transmission based on multi-node adaptive parametric tuning according to this application. Detailed Implementation

[0019] This invention provides a method for image recoding and transmission based on multi-node adaptive parameter tuning. It is mainly applied to the recoding and transmission of non-standard encoded bitstreams in real-time use scenarios with high requirements for network bandwidth, image frame rate, and low latency. The core of this method is to read YUV422 image data from shared memory, and then use a multi-node thread group to encode the original YUV422 image sequentially using a polling method and a custom image encoding. By performing parallel image encoding on the node thread group, the method can adaptively adjust the number of node thread groups and the image recoding coefficient factor according to the current computer resources, the number of image channels, and the image recoding coefficient factor. The image is then encoded into JPEG and transmitted, enabling strong temporal, high frame rate, low latency, and continuous transmission of image sets with optimal network bandwidth utilization and image frame rate. like Figure 1 As shown, a method for image recoding and transmission based on multi-node adaptive parametric tuning is described, with the following steps: S1. Utilize a node thread group number adjustment factor constructed using a custom set of computer resource parameters, the number of image set channels, and the data relationship between node thread groups to obtain the number of node thread groups for the current image channel. The computer resource parameter set includes the computer's network bandwidth, number of computer cores, computer clock speed, and computer memory.

[0020] The number of image set channels refers to the total number of image channels currently being processed by the computer. For example, if one airplane has 1 visible light video and one airplane has 1 infrared video, then the number of image set channels is 2, and so on.

[0021] The number of node thread groups refers to the number of concurrent threads currently processing concurrent data.

[0022] 11) Calculate the node thread group number adjustment factor. : ; in, Indicates standardized network bandwidth. ; This represents the standardized number of computer CPU cores. ; This refers to the standardized computer clock speed. ; This represents standardized memory bandwidth. Represents the standardized number of image channels. ; , , as well as These are all parameters from the computer resource parameter set; express The adjustment factor, and so on; Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was calculated using the least squares method. The initial value; The calculation yielded Initial values ​​include: By minimizing the sum of squared errors The unique analytic value of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial adjustment factors obtained are as follows: ; 12) Obtain the independent variable matrix in real time During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; 13) Adjust the factor based on the obtained number of node thread groups. The number of node thread groups was calculated.

[0023] ; Where N represents the number of node thread groups, and the subscript c represents the number of image channels.

[0024] S2. By utilizing the data relationship between the custom node thread group number adjustment factor and the image set frame rate, an image adaptive adjustment factor is constructed, and the image recoding coefficient factor is obtained. Here, the image set frame rate represents the frame rate of a specific image channel dataset.

[0025] 21) Image adaptive adjustment factor for: ; in, The adjustment factor for the number of node thread groups is... , The frame rate of the standardized image set is... ,in, ; express The adjustment factor, and so on.

[0026] Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was solved using the least squares method, resulting in... The initial value; the calculation obtained Initial values ​​include: By minimizing the sum of squared errors The unique analytic value of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial factor values ​​obtained are as follows: ; 22) Real-time acquisition of independent variable matrix During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; 23) Adjust the image adaptively. Image recoding coefficient factor

[0027] .

[0028] S3. Timestamp the original YUV422 image in the shared memory to form an image pre-coding dataset and put it into the resource pool.

[0029] The resource pool refers to the cache queue inside the program.

[0030] S4. The multi-node thread group extracts the corresponding image precoding dataset from the resource pool according to the strong temporal dataset and marks the node thread group encoding.

[0031] The node thread group encoding includes the node number and the timestamp number in the image precoding dataset.

[0032] S5. Each node thread group re-encodes the YUV422 images in the image precoding dataset encoded by the marked node thread group according to the image recoding coefficient factor to form a JPEG image set. Each node thread group normalizes the image recoding coefficient factor solved in S2.2 into an image coding quality factor, and encodes the YUV422 images into JPEG images to form a JPEG image set.

[0033] The image encoding quality factor is the quality factor for YUV422 images encoded as JPEG images.

[0034] S6. Each node thread group performs strong temporal correlation framing processing on the re-encoded JPEG image set to obtain the framed image dataset. Each node thread group performs temporal correlation framing according to the custom frame format and the temporal order of the JPEG images in the JPEG image set to obtain the framed image dataset.

[0035] Among them, frame assembly processing refers to combining the encoded image data, frame order, and variable task information data into a complete frame according to a custom format.

[0036] The custom frame format is shown in the figure below, where B represents bytes.

[0037]

[0038] Wherein, (1) represents the frame header synchronization word; (2) represents the image channel index; (3) represents the image width; (4) represents the image height; (5) represents the current frame number; (6) represents the task attribute length; (7) represents the reserved field; (8) represents the data area frame length; (9) represents the data area; (10) represents the frame tail.

[0039] S7. After performing sliding window sorting on the framed image dataset using a strong timing polling timing window, Websocket forwarding is then performed.

[0040] By periodically polling to trigger the updating and calculation of the sliding window, real-time and continuous processing of image datasets is achieved, specifically including: Set polling period Window size Sliding step size ,in, Image frame rate; After performing strong temporal sorting on the image dataset retrieved by the sliding window according to the parameters set above, it is then forwarded via WebSocket.

[0041] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention based on the above-disclosed technical content without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A method for image recoding and transmission based on multi-node adaptive parametric tuning, characterized in that, include: The number of node thread groups for the current image channel is obtained by constructing a node thread group number adjustment factor using a custom set of computer resource parameters, the data relationship between the number of image channels and the number of node thread groups; By utilizing the data relationship between a custom node thread group number adjustment factor and the image set frame rate, an image adaptive adjustment factor is constructed, resulting in the image recoding coefficient factor; The original YUV422 images in shared memory are timestamped to form an image precoding dataset, which is then placed into the resource pool. The multi-node thread group extracts the corresponding image precoding dataset from the resource pool according to the strong temporal dataset and marks the node thread group encoding; Each node thread group re-encodes the YUV422 images in the image precoding dataset encoded by the marked node thread group according to the image recoding coefficient factor to form a JPEG image set; Each node thread group performs strong temporal correlation frame assembly processing on the re-encoded JPEG image set to obtain the framed image dataset; After the framed image dataset is sorted by a strong timing polling timing window, it is then forwarded via WebSocket.

2. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 1, characterized in that: The node thread group number adjustment factor, constructed using a custom set of computer resource parameters and the data relationship between the number of image channels and the number of node thread groups, is used to obtain the current number of node thread groups for the image channel, including: Set the node thread group number adjustment factor : ； in, Indicates standardized network bandwidth. ; This represents the standardized number of computer CPU cores. ; This refers to the standardized computer clock speed. ; This represents standardized memory bandwidth. Represents the standardized number of image channels. ; , , as well as These are all parameters from the computer resource parameter set; express The adjustment factor, and so on; Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was calculated using the least squares method. The initial value; The calculation yielded Initial values ​​include: By minimizing the sum of squared errors The unique analytic value of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial adjustment factors obtained are as follows: ; Real-time acquisition of independent variable matrix During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; Adjust the factor based on the obtained number of node thread groups. The number of node thread groups was calculated. ； Where N represents the number of node thread groups, and the subscript c represents the number of image channels.

3. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 2, characterized in that: The image adaptive adjustment factor, constructed by utilizing the data relationship between a custom node thread group number adjustment factor and the image set frame rate, yields the image recoding coefficient factor, including: Let the image adaptive adjustment factor be set. for: ； in, The node thread group number adjustment factor is... , The frame rate of the standardized image set is... ,in, ; express The adjustment factor, and so on; Let the set of adjustment factors be related as follows: In this process, by collecting more than 200 sets of data, the optimal adjustment factor was solved using the least squares method, resulting in... The initial value; Real-time acquisition of independent variable matrix During the process, after every 20 new sets of measured data, the adjustment factor is iteratively updated using the least squares method; Based on image adaptive adjustment factor Image recoding coefficient factor 。 4. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 3, characterized in that: The calculation yielded Initial values ​​include: By minimizing the sum of squared errors The unique analytic value of the factor is obtained after differentiation. ,in, The independent variable matrix is The initial factor values ​​obtained are as follows: .

5. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 1, characterized in that: The node thread group encoding includes the node number and the timestamp number in the image pre-encoding dataset.

6. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 1, characterized in that: Each node thread group re-encodes the YUV422 images in the image precoding dataset encoded by the marked node thread group to form a JPEG image set based on the image recoding coefficient factor. This includes: each node thread group normalizes the solved image recoding coefficient factor into an image coding quality factor, encodes the YUV422 images into JPEG images, and forms a JPEG image set.

7. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 1, characterized in that: Each node thread group performs strong temporal association framing processing on the re-encoded JPEG image set to obtain a framed image dataset, including: each node thread group performs temporal association framing according to a custom frame format and the temporal order of the JPEG images in the JPEG image set to obtain a framed image dataset.

8. The method for image recoding and transmission based on multi-node adaptive parametric tuning according to claim 1, characterized in that: The process of using a strong-time polling timing window to perform sliding window sorting on the framed image dataset before WebSocket forwarding includes: By periodically polling to trigger the updating and calculation of the sliding window, real-time and continuous processing of image datasets is achieved, specifically including: Set polling period Window size sliding step size ,in, Image frame rate; After performing strong temporal sorting on the image dataset retrieved by the sliding window according to the parameters set above, it is then forwarded via WebSocket.

Images