Image processing method and system

Abstract

The invention provides an image processing method and system, and the method comprises the steps: adjusting a parameter value of at least one configuration parameter according to a first control signal and a second control signal, and obtaining a current value of the at least one configuration parameter; the first control signal is generated by monitoring a VGA write video memory bus according to a monitoring result; the second control signal is generated according to a monitoring result by monitoring the response time of the output write operation reply; acquiring image data according to the current value of the capture frame rate; performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; compressing the data processing result by adopting the corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and outputting the compressed data. The frame rate, the quantization table and the YUV data output format are adjusted by detecting the VGA write video memory bus and the output response time of the write operation reply, and the resource utilization rate is high.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and system. Background Art

[0002] Baseboard Manager Controller (BMC) is usually used in the server field. The chip and the system software running on it are used to monitor the status of the server (temperature, fan, main CPU operation status, etc.). At the same time, another important function is to transmit local image information to the remote end through the network for remote display and monitoring, so as to facilitate remote observation and operation by users.

[0003] In the prior art, in the process of transmitting images, if the original data is transmitted, the data volume is huge, which affects the transmission efficiency. Generally, the image is compressed before transmission. In the prior art, the image processing module obtains the original image data according to the configuration set by the driver, and then compresses the image data through data format conversion, YUV sampling, BLOCK conversion, JPEGIP (DCT transformation, quantization, encoding), and finally writes the processed compressed data to DDR through the output module, and outputs each frame of image information, including frame header, frame tail, frame size, etc. The driver reads this information and passes it to the network, gets the correct image data from DDR, and displays it at the remote end.

[0004] In this process, the image processing module can only perform image processing according to the configuration parameters set by the driver (including capture mode, frame rate, YUV data output format, quantization table and other parameters), which has the disadvantage of wasting DDR memory and DDR bus bandwidth. Especially when the DDR bandwidth is insufficient, it will compete with other modules and affect each other. For example, the image processing module is configured to obtain data through the AXI bus, the frame rate is 60fp / s, the YUV data output format is YUV444, and the default quantization table is used. If the bandwidth is insufficient, the image processing module will compete with the VGA (Video Graphics Array, video transmission standard) module, resulting in VGA operation jams, display screen distortion and other phenomena. When users find this situation, they can only modify the configuration parameters in the driver again to reduce the image processing module's occupancy of DDR, which is inefficient.

[0005] In summary, the existing technology has the problem of low computing resource utilization. Summary of the invention

[0006] The present invention provides an image processing method and system, which are used to solve the defect of low computing resource utilization in the prior art and realize image processing with high computing resource utilization.

[0007] The present invention provides an image processing method, comprising: The parameter value of at least one configuration parameter is adjusted according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result of monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result of monitoring the response time of the output write operation reply; acquiring image data according to a current value of the capture frame rate; Performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; Compressing the data processing result using a corresponding quantization table according to a current value of the quantization table category to obtain compressed data; The compressed data is output.

[0008] According to an image processing method provided by the present invention, a VGA write memory bus is monitored, and a first control signal is generated according to the monitoring result, specifically comprising: When the VGA write memory bus is operated and the write address falls within the current memory range, generating a detection valid signal; When the number of the detection effective signals in one frame of image data is greater than a first preset number threshold, a detection effective output signal is generated as a first control signal.

[0009] According to an image processing method provided by the present invention, monitoring the response time of the output write operation reply and generating a second control signal according to the monitoring result specifically includes: If the current output write operation is valid, record the first time it takes to receive the reply to the current write operation; When the first time is not less than a preset time threshold, a bandwidth shortage flag signal is output as a second control signal.

[0010] According to an image processing method provided by the present invention, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the detection effective output signal, calculating the image update frequency according to the number of the detection effective output signals acquired within a preset time range; In the case where the image update frequency is greater than a first preset update threshold, detecting a current capture frame rate; if the current capture frame rate is less than a first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; In the case where the image update frequency is not greater than a first preset update threshold and greater than a second preset update threshold, detecting a current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; When the image update frequency is not greater than a second preset update threshold, the current capture frame rate is determined according to the number of times the valid output signal is detected per second.

[0011] According to an image processing method provided by the present invention, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the insufficient bandwidth flag signal, recording a target number of receiving the insufficient bandwidth flag signal within a frame of image data; When the target quantity is not less than a second preset quantity threshold, detecting a current capture frame rate; If the current capture frame rate is greater than a third preset frame rate threshold, executing a first adjustment strategy; If the current capture frame rate is not greater than a fourth preset frame rate threshold, the current YUV data output format is detected and a second adjustment strategy is executed.

[0012] According to an image processing method provided by the present invention, the first adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until a second bandwidth shortage flag signal is obtained; In response to obtaining the second bandwidth shortage flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth shortage flag signal is obtained; In response to acquiring the third insufficient bandwidth flag signal, the current YUV data output format is detected and a second adjustment strategy is executed.

[0013] According to an image processing method provided by the present invention, the second adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; If the current YUV data output format is the first preset data format, the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until a second bandwidth insufficient flag signal is obtained; In response to acquiring the second bandwidth insufficient flag signal, detecting the current quantization table and executing a third adjustment strategy; If the current YUV data output format is the second preset data format, detecting the current quantization table and executing a third adjustment strategy; Wherein, the third strategy includes: If the current quantization table is a high-quality quantization table, adjusting the current quantization table to a default quantization table; If the current quantization table is a default quantization table, the current quantization table is adjusted to a low-quality quantization table.

[0014] The present invention also provides an image processing system, comprising: A monitoring unit, used for monitoring the VGA write memory bus and generating a first control signal according to the monitoring result; A detection unit, configured to generate the second control signal based on monitoring of a response time of a reply to an output write operation by a monitoring module and based on a monitoring result; A control unit, configured to adjust a parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain a current value of the at least one configuration parameter; wherein the configuration parameters include a capture frame rate, a YUV data output format, and a quantization table category; a data acquisition unit, configured to acquire image data according to a current value of the capture frame rate; A format processing unit, used for performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; A compression unit, configured to compress the data processing result using a corresponding quantization table according to a current value of the quantization table category to obtain compressed data; An output unit is used to output the compressed data.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned image processing methods is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image processing method described above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method described above is implemented.

[0018] The image processing method and system provided by the present invention adjust the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result by monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result by monitoring the response time of the output write operation reply; image data is obtained according to the current value of the capture frame rate; data processing is performed on the image data according to the current value of the YUV data output format to obtain data processing results; the data processing results are compressed using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and the compressed data is output. The present invention generates a control signal to automatically adjust the frame rate, quantization table and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is one of the flowchart diagrams of the image processing method provided by the present invention.

[0021] Figure 2 It is a schematic diagram of an application scenario of the image processing method provided by the present invention.

[0022] Figure 3 This is the second flowchart of the image processing method provided by the present invention.

[0023] Figure 4 This is the third flow chart of the image processing method provided by the present invention.

[0024] Figure 5 It is a structural schematic diagram of the image processing system provided by the present invention.

[0025] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Combine the following Figure 1-Figure 4 The image processing method of the present invention is described. Figure 1 It is one of the structural schematic diagrams of the image processing method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: The present invention provides an image processing method, comprising the following steps: Step 110: adjusting the parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result of monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result of monitoring the response time of the output write operation reply; Step 120: Acquire image data according to the current value of the capture frame rate; Step 130: performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; Step 140: compressing the data processing result using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; Step 150: Output the compressed data.

[0028] It should be explained that the application scenarios of the present invention include using the BMC chip to transmit local image information to the remote end through the network for remote display and monitoring. It should be emphasized that the present invention does not impose any specific restrictions on the specific chips used. In one embodiment, the application scenario is as follows: Figure 2As shown in the figure, specifically, the information on the HOST side is transmitted to the VGA module inside the BMC chip through PCIE. The VGA module interacts with DDR, and is processed and converted into DVI (Digital Visual Interface) interface signals, which are displayed on the local display. If you want to display on a remote display, you can obtain image data from DDR (Double Data Rate) through the DVI interface or AXI (Advanced eXtensible Interface) bus, and then compress the image data through the image processing module, write the compressed image data to DDR, and finally obtain the processed data in DDR through the network and transmit it to the remote display, thereby realizing the function of remote management and control.

[0029] Among them, the AXI bus is an on-chip bus for high performance, high bandwidth, and low latency, including master devices, slave devices, bus interconnection parts, etc., and is a multi-master and multi-slave structure.

[0030] Since the image processing module of the prior art can only perform image processing according to fixed configuration parameters, there is a problem of waste of computing resources, especially waste of DDR memory and DDR bus bandwidth. The present invention adds a monitoring module, a control module and a monitoring module on the basis of completing the modules required for the necessary image processing flow, namely, a data acquisition module, a format processing module, a compression module and an output module.

[0031] The monitoring module is used to implement the monitoring function, and determines the frequency of updating the current video memory content by monitoring the signal on the VGA write video memory bus (AXI write channel), thereby generating a first control signal to determine the frame rate at which the data acquisition module acquires data.

[0032] The monitoring module is used to monitor the response time of the DDR reply when the output module writes to the DDR, and judge the bandwidth situation at this time according to the response time, so as to generate a second control signal to determine the frame rate of the data acquired by the data acquisition module and / or the YUV data output format of the format processing module and / or the quantization table category adopted by the compression module.

[0033] The control module adopts different strategies to reduce the time that the image processing module occupies DDR according to the results of the listening module and the monitoring module (i.e., the first control signal and the second control signal), including adjusting the capture frame rate, the YUV data output format, and using quantization tables of different quantization table categories (i.e., compression quality), to obtain the current value of each configuration parameter. It should be noted that in each adjustment, the three configuration parameters are not necessarily adjusted at the same time, and the current value of the configuration parameter may be the updated value after the adjustment, or it may be the historical value before the adjustment.

[0034] Furthermore, a RAM module can be added. The RAM module is used to store quantization tables of different compression qualities, including quantization tables of three different quantization table categories: high quality, medium (default), and low quality. It can be understood that the higher the compression quality, the more compressed image data, and the lower the compression quality, the less compressed image data. The present invention uses RAM to store the quantization table, which avoids operating a large number of registers when changing the quantization table, improves the speed, and is simple to operate.

[0035] The data acquisition module is used to acquire image data according to the current value of the capture frame rate; Figure 3 As shown, the monitoring module can be set in the data acquisition module.

[0036] The format processing module is used to process data according to the current value of the YUV data output format to obtain a data processing result. In some embodiments, the format processing module includes an RGB2YUV module and a YUV2BLOCK module, such as Figure 3 shown.

[0037] The compression module is used to compress the data processing result using the corresponding quantization table according to the current value of the quantization table category to obtain compressed data. In some embodiments, the compression module includes a JPEG IP module, such as Figure 3 shown.

[0038] The output module is used to output compressed data.

[0039] The present invention realizes automatic adjustment of the frame rate, avoids the disadvantages of traditional solutions, reduces the waste of DDR memory, improves the utilization rate of DDR bus bandwidth, optimizes the overall performance of the chip, and improves the user experience of the remote end.

[0040] The generation of the first control signal is further described below. In some embodiments, monitoring the VGA write memory bus and generating the first control signal according to the monitoring result specifically includes: When the VGA write memory bus is operated and the write address falls within the current memory range, generating a detection valid signal; When the number of the detection effective signals in one frame of image data is greater than a first preset number threshold, a detection effective output signal is generated as a first control signal.

[0041] Specifically, the monitoring module monitors the VGA write memory bus (i.e., the AXI write channel on the VGA). When an operation is detected on the AXI write channel on the VGA, it determines whether the write address falls within the current memory range. If it falls within the current memory range, it indicates that there is data update in the memory.

[0042] It should be explained that the video memory range at this time is: video memory base address~video memory base address+address space size occupied by the current pixel; the address space size occupied by the current pixel is determined according to different bpp modes, and the present invention does not limit this.

[0043] In one embodiment, the bpp mode is 32bpp, and a pixel value occupies 32 bits, that is, 4 bytes. Therefore, the address space size occupied by the current pixel = the number of pixels * 4.

[0044] In another embodiment, the bpp mode is 16bpp, and a pixel value occupies 16 bits, that is, 2 bytes. Therefore, the address space occupied by the current pixel = the number of pixels * 2

[0045] Furthermore, when it is detected that the data in the video memory is updated, a detection valid signal is generated. Afterwards, the number of detection valid signals in a frame of image data is recorded. When the number is greater than a first preset number threshold, a detection valid output signal is triggered, and the first control signal is the detection valid output signal. Otherwise, the detection valid output signal is not triggered, and the first control signal is empty.

[0046] Furthermore, the present invention does not impose any specific limitation on the first preset quantity threshold. In some embodiments, the first preset quantity threshold is 1.

[0047] It should be noted that the present invention does not limit the method of recording the number of effective signals detected in a frame of image data, and the recording method can be selected according to actual conditions. In one embodiment, a counter cnt1 is set, and the counter value is increased by 1 each time a valid write operation is detected.

[0048] Based on the above embodiment, when the number of write operations detected within a frame of image data (the value of cnt1) is greater than or equal to the set threshold 1, it is considered that the current frame of image data has changed, and the detection of a valid output signal is triggered; when the number of write operations detected within a frame of image data (the value of cnt1) is less than the set threshold 1, it is considered that the current frame of image data has not changed, and the detection of a valid output signal is not triggered.

[0049] The generation of the second control signal is further explained below. In some embodiments, monitoring the response time of the output write operation reply and generating the second control signal according to the monitoring result specifically includes: If the current output write operation is valid, record the first time it takes to receive the reply to the current write operation; When the first time is not less than a preset time threshold, a bandwidth shortage flag signal is output as a second control signal.

[0050] Specifically, the monitoring module monitors the AXI write channel of the output module writing DDR, and when the output module initiates a valid write operation, that is, when awvalid and awready are both high, the first time of receiving a reply to the current write operation is recorded.

[0051] Then the first time is judged. If the first time is greater than or equal to the preset time threshold, it indicates that the bandwidth may be insufficient at this time, and a bandwidth shortage flag signal is output. At this time, the second control signal is the bandwidth shortage flag signal. On the contrary, if the first time is less than the preset time threshold, it indicates that the current DDR bandwidth is sufficient for the image processing module to use, and no adjustment is required, and no flag signal is output. At this time, the second control signal is empty.

[0052] Furthermore, the present invention does not impose any specific limitation on the preset time threshold. In some embodiments, the preset time threshold is 100 clock cycles.

[0053] It should be noted that the present invention does not limit the method of recording the first time of receiving the reply of the current write operation, and the recording method can be selected according to the actual situation. In one embodiment, a counter cnt3 is set, and the counter value is increased by 1 every clock cycle until a reply of the write operation is received, that is, bvalid and bready are high at the same time, and then the counting stops.

[0054] Based on the above embodiment, if the counter value cnt3 at this time is less than the set threshold value 3, it indicates that the current DDR bandwidth is sufficient for the image processing module, no adjustment is required, and no flag signal is output; if the counter value cnt3 at this time is greater than or equal to the set threshold value 3, it indicates that the bandwidth at this time may be insufficient, and a flag signal is output.

[0055] The adjustment of the configuration parameters is further described below. In some embodiments, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the detection effective output signal, calculating the image update frequency according to the number of the detection effective output signals acquired within a preset time range; In the case where the image update frequency is greater than a first preset update threshold, detecting a current capture frame rate; if the current capture frame rate is less than a first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; In the case where the image update frequency is not greater than a first preset update threshold and greater than a second preset update threshold, detecting a current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; When the image update frequency is not greater than a second preset update threshold, the current capture frame rate is determined according to the number of times the valid output signal is detected per second.

[0056] Specifically, the detection effective output signal is transmitted to the control module, the number of the detection effective output signals obtained within a preset time range is recorded, and the image update frequency is calculated according to the preset time range and the number of the detection effective output signals.

[0057] It should be noted that the present invention does not limit the method of recording the number of effective detection output signals obtained within the preset time range, and the recording method can be selected according to the actual situation. In one embodiment, a counter cnt2 is set, and each time a valid detection output signal of the monitoring module is received, the counter value is increased by 1, and the frequency of image updates within 1 second is counted to obtain the image update frequency. That is, in this embodiment, the preset time range is 1 second.

[0058] Afterwards, the control module adjusts the current working frame rate (ie, the capture frame rate) according to the image update frequency.

[0059] It should be noted that the present invention may also include a frame rate adjustment module. The control module controls the frame rate adjustment module to adjust the capture frame rate according to the image update frequency. The present invention does not limit the specific adjustment method and module details.

[0060] During the adjustment process, when the image update frequency is greater than the first preset update threshold, it indicates that the image is updated relatively quickly, and at this time the current capture frame rate is detected.

[0061] The present invention does not impose any specific restrictions on the first preset update threshold. In a specific embodiment, the first preset update threshold is 30 image updates within 1 second.

[0062] Then, the current capture frame rate is judged. If the current capture frame rate is less than the first preset frame rate threshold, the current capture frame rate is adjusted to the first frame rate to achieve a better display effect. On the contrary, if the current capture frame rate is not less than the first preset frame rate threshold, the image data is captured according to the set frame rate, that is, the current capture frame rate is kept unchanged. It should be noted that the first preset frame rate threshold is less than the first frame rate, and the present invention does not impose other restrictions on the first preset frame rate threshold and the first frame rate.

[0063] In a specific embodiment, the first preset frame rate threshold is 30 frames, and the first frame rate is 45 frames.

[0064] When the image update frequency is not greater than the first preset update threshold and greater than the second preset update threshold, it indicates that the image update is slow, and the current capture frame rate is detected. The present invention does not impose specific restrictions on the second preset update threshold. In a specific embodiment, the second preset update threshold is 10 image updates within 1 second.

[0065] Then, the current capture frame rate is judged. If the current capture frame rate is not less than the first preset frame rate threshold, the current capture frame rate is adjusted to the second frame rate to reduce the transmission of invalid data and save bandwidth. On the contrary, if the current capture frame rate is less than the first preset frame rate threshold, the image data is captured according to the set frame rate, that is, the current capture frame rate is kept unchanged.

[0066] It should be noted that the second frame rate is lower than the first preset frame rate, and the first preset frame rate is lower than the first frame rate. The present invention does not impose other restrictions on the second frame rate, the first preset frame rate and the first frame rate.

[0067] In a specific embodiment, the first frame rate is 45 frames, the first preset frame rate threshold is 30 frames, and the second frame rate is 20 frames.

[0068] When the image update frequency is not greater than the second preset update threshold, it indicates that the image is updated very slowly, then the current capture mode is changed, and the work is started in the monitoring mode, that is, the monitoring module starts working only when it triggers the detection of the valid output signal. Specifically, the current capture frame rate is determined according to the number of times the valid output signal is detected per second, that is, when the image update frequency is not greater than the second preset update threshold, at this time, the monitoring mode is used, and the set capture frame rate does not work. The current capture frame rate is determined according to the number of valid output signals to capture. For example, if the image is updated twice in 1 second, that is, 2 valid output signals are output, then only two frames are captured in 1 second. If the image is updated 7 times in 1 second, that is, 7 valid output signals are output, then only 7 frames are captured in 1 second. The present invention obtains updated image data from DDR through the AXI bus, avoids the transmission of invalid data, and saves DDR memory and bandwidth.

[0069] Furthermore, in some embodiments, the adjusting of the configuration parameters further includes: adjusting the configuration parameters according to the first control signal and the second control signal, specifically including: In response to acquiring the insufficient bandwidth flag signal, recording a target number of receiving the insufficient bandwidth flag signal within a frame of image data; When the target quantity is not less than a second preset quantity threshold, detecting a current capture frame rate; If the current capture frame rate is greater than a third preset frame rate threshold, executing a first adjustment strategy; If the current capture frame rate is not greater than a fourth preset frame rate threshold, the current YUV data output format is detected and a second adjustment strategy is executed.

[0070] Specifically, the insufficient bandwidth flag signal is transmitted to the control module, and the target number of insufficient bandwidth flag signals received within one frame of image data is recorded.

[0071] The control module adjusts the configuration parameters of the current operation according to the target number to ensure that the relevant modules can work normally. If the target number is less than the second preset number threshold, it indicates that the bandwidth can meet the image processing module to work in the current working mode and no adjustment is required. If the target number is greater than or equal to the set second preset number threshold, it indicates that the bandwidth cannot meet the image processing module to work in the current working mode and the corresponding parameters need to be adjusted. At this time, the current capture frame rate is detected.

[0072] It should be noted that the present invention does not limit the method of recording the number of effective signals detected in a frame of image data, and the recording method can be selected according to actual conditions. Based on the above embodiment, a counter cnt4 is set to count the number of times the cnt3 value exceeds the threshold 3 (i.e., the bandwidth insufficient flag signal is issued) during the output of a frame of image data. It should be noted that after adjusting the adjustment parameters, the counter cnt4 value is reset to zero.

[0073] In one embodiment, the second preset number threshold is 100.

[0074] The adjustment strategy of the configuration parameters is further described below. During the adjustment process, the detected current capture frame rate is judged. If the current capture frame rate is greater than the third preset frame rate threshold, the first adjustment strategy is executed; if the current capture frame rate is less than or equal to the fourth preset frame rate threshold, the capture frame rate is not adjusted, the current YUV data output format is detected and the second adjustment strategy is executed; if the current capture frame rate is not greater than the third preset frame rate threshold and not less than the fourth preset frame rate threshold, the capture frame rate is adjusted to the fourth preset frame rate threshold.

[0075] It should be noted that the third preset frame rate threshold is greater than the fourth preset frame rate threshold, and the present invention does not impose other restrictions on the third preset frame rate threshold and the fourth preset frame rate threshold. In a specific embodiment, the third preset frame rate threshold is 30 frames per second, and the fourth preset frame rate threshold is 20 frames per second.

[0076] Furthermore, in some embodiments, the first adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until a second bandwidth shortage flag signal is obtained; In response to obtaining the second bandwidth shortage flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth shortage flag signal is obtained; In response to acquiring the third insufficient bandwidth flag signal, the current YUV data output format is detected and a second adjustment strategy is executed.

[0077] Specifically, for the convenience of distinction, the currently received insufficient bandwidth flag signal is recorded as the first insufficient bandwidth flag signal, and then the current capture frame rate is judged. If the detected current capture frame rate is greater than the third preset frame rate threshold, the current capture frame rate is adjusted to the third preset frame rate threshold, and the bandwidth situation is continued to be monitored. If the bandwidth is satisfied (that is, the second insufficient bandwidth flag signal is not obtained), the current capture frame rate is maintained at the third preset frame rate threshold; if it is still insufficient (that is, the second insufficient bandwidth flag signal is obtained), the current capture frame rate is adjusted to the fourth preset frame rate threshold, and the bandwidth situation is continued to be monitored. If the bandwidth is satisfied (that is, the third insufficient bandwidth flag signal is not obtained), the current capture frame rate is maintained at the fourth preset frame rate threshold; if it is still insufficient (that is, the second insufficient bandwidth flag signal is obtained), the second adjustment strategy is executed.

[0078] Based on the above embodiment, the third preset frame rate threshold is 30 frames per second, and the fourth preset frame rate threshold is 20 frames per second. Then, the first strategy includes: if the frame rate is greater than 30, the data acquisition frame rate is adjusted to 30 frames per second, and the bandwidth situation is continuously monitored; if the bandwidth is sufficient, the frame rate is maintained at 30 frames per second; if it is still insufficient, the data acquisition frame rate is adjusted to 20 frames per second, and the bandwidth situation is continuously monitored; if the bandwidth is sufficient, the frame rate is maintained at 20 frames per second; if it is still insufficient, the second adjustment strategy is executed.

[0079] Furthermore, in some embodiments, the second adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; If the current YUV data output format is the first preset data format, the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until a second bandwidth insufficient flag signal is obtained; In response to acquiring the second bandwidth insufficient flag signal, detecting the current quantization table and executing a third adjustment strategy; If the current YUV data output format is the second preset data format, detecting the current quantization table and executing a third adjustment strategy; Wherein, the third strategy includes: If the current quantization table is a high-quality quantization table, adjusting the current quantization table to a default quantization table; If the current quantization table is a default quantization table, the current quantization table is adjusted to a low-quality quantization table.

[0080] Specifically, for the convenience of distinction, the currently received insufficient bandwidth flag signal is recorded as the first insufficient bandwidth flag signal, and then the current YUV data output format is judged. If the current YUV data output format is the first preset data format, it is adjusted to the second preset data format, and the bandwidth situation is continuously monitored. If the bandwidth is satisfied (i.e., the second insufficient bandwidth flag signal is not obtained), the YUV data output format is maintained as the second preset data format. If it is still insufficient (i.e., the second insufficient bandwidth flag signal is obtained), the current quantization table is detected and the third strategy is executed. On the contrary, if the current YUV data output format is the second preset data format, no adjustment is made, and the current quantization table is detected and the third strategy is executed.

[0081] In a specific embodiment, the first preset data format is YUV444, and the second preset data format is YUV420.

[0082] Among them, the third strategy includes: if the current quantization table is a high-quality quantization table, it can be adjusted to a default (medium) quantization table; if the current quantization table is a default quantization table, it can be adjusted to a low-quality quantization table; if the current quantization table is a low-quality quantization table, no adjustment is made.

[0083] The present invention adopts the mode of adaptively adjusting the frame rate, YUV data output format and quantization table, thereby avoiding the transmission of invalid data, saving DDR memory, improving the utilization rate of DDR bus bandwidth and reducing the demand for bandwidth.

[0084] In addition, for the multiple thresholds mentioned in the present invention, the user can configure different thresholds according to needs to control the display effect.

[0085] Based on the above embodiments, in one embodiment, the internal structure of the control module is as follows: Figure 4 shown.

[0086] Among them, the counting module 1 counts the effective detection signals transmitted by the monitoring module, with 1 second as the time unit, and then transmits the statistical value to the comparison module 1. The comparison module 1 compares the statistical quantity and the working data acquisition frame rate accordingly, and adjusts the frame rate of the data acquisition module through the frame rate adjustment module according to different comparison results, so as to achieve the purpose of saving DDR memory and bandwidth; The counting module 2 counts the bandwidth shortage flag signal transmitted from the monitoring module, with one frame of image as the time unit, and then transmits the statistical value to the comparison module 2. The comparison module 2 compares the statistical value with the data acquisition frame rate working at this time, and adjusts the frame rate of the data acquisition module through the frame rate adjustment module according to different comparison results, so as to save DDR memory and bandwidth; if the bandwidth is still insufficient after adjusting the frame rate, the strategy of adjusting the YUV data output format (the second strategy) is used; if the bandwidth is still insufficient after adjusting the YUV data output format, the strategy of adjusting the quantization table (the third strategy) is used.

[0087] The image processing method provided by the present invention adjusts the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result by monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result by monitoring the response time of the output write operation reply; image data is acquired according to the current value of the capture frame rate; data processing is performed on the image data according to the current value of the YUV data output format to obtain data processing results; the data processing results are compressed using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and the compressed data is output. The present invention generates a control signal to automatically adjust the frame rate, quantization table and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization.

[0088] The image processing system provided by the present invention is described below. The image processing system described below and the image processing method described above can be referenced to each other. Figure 5 Schematic diagram of the structure of the image processing system provided by the present invention. Figure 5 As shown, the system includes the following modules: A monitoring unit 510 is used to monitor the VGA write memory bus and generate a first control signal according to the monitoring result; A detection unit 520, configured to generate the second control signal based on monitoring of a response time of a write operation reply output by a monitoring module and based on a monitoring result; A control unit 530, configured to adjust a parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain a current value of the at least one configuration parameter; wherein the configuration parameters include a capture frame rate, a YUV data output format, and a quantization table category; A data acquisition unit 540, configured to acquire image data according to a current value of the capture frame rate; A format processing unit 550, configured to perform data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; A compression unit 560, configured to compress the data processing result using a corresponding quantization table according to a current value of the quantization table category to obtain compressed data; The output unit 570 is configured to output the compressed data.

[0089] According to an image processing system provided by the present invention, a VGA write memory bus is monitored, and a first control signal is generated according to the monitoring result, specifically including: When the VGA write memory bus is operated and the write address falls within the current memory range, generating a detection valid signal; When the number of the detection effective signals in one frame of image data is greater than a first preset number threshold, a detection effective output signal is generated as a first control signal.

[0090] According to an image processing system provided by the present invention, monitoring the response time of the output write operation reply and generating a second control signal according to the monitoring result specifically includes: If the current output write operation is valid, record the first time it takes to receive the reply to the current write operation; When the first time is not less than a preset time threshold, a bandwidth shortage flag signal is output as a second control signal.

[0091] According to an image processing system provided by the present invention, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the detection effective output signal, calculating the image update frequency according to the number of the detection effective output signals acquired within a preset time range; In the case where the image update frequency is greater than a first preset update threshold, detecting a current capture frame rate; if the current capture frame rate is less than a first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; In the case where the image update frequency is not greater than a first preset update threshold and greater than a second preset update threshold, detecting a current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; When the image update frequency is not greater than a second preset update threshold, the current capture frame rate is determined according to the number of times the valid output signal is detected per second.

[0092] According to an image processing system provided by the present invention, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the insufficient bandwidth flag signal, recording a target number of receiving the insufficient bandwidth flag signal within a frame of image data; When the target quantity is not less than a second preset quantity threshold, detecting a current capture frame rate; If the current capture frame rate is greater than a third preset frame rate threshold, executing a first adjustment strategy; If the current capture frame rate is not greater than a fourth preset frame rate threshold, the current YUV data output format is detected and a second adjustment strategy is executed.

[0093] According to an image processing system provided by the present invention, the first adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until a second bandwidth shortage flag signal is obtained; In response to obtaining the second bandwidth shortage flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth shortage flag signal is obtained; In response to acquiring the third insufficient bandwidth flag signal, the current YUV data output format is detected and a second adjustment strategy is executed.

[0094] According to an image processing system provided by the present invention, the second adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; If the current YUV data output format is the first preset data format, the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until a second bandwidth insufficient flag signal is obtained; In response to acquiring the second bandwidth insufficient flag signal, detecting the current quantization table and executing a third adjustment strategy; If the current YUV data output format is the second preset data format, detecting the current quantization table and executing a third adjustment strategy; Wherein, the third strategy includes: If the current quantization table is a high-quality quantization table, adjusting the current quantization table to a default quantization table; If the current quantization table is a default quantization table, the current quantization table is adjusted to a low-quality quantization table.

[0095] The image processing system provided by the present invention adjusts the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result by monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result by monitoring the response time of the output write operation reply; image data is acquired according to the current value of the capture frame rate; data processing is performed on the image data according to the current value of the YUV data output format to obtain data processing results; the data processing results are compressed using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and the compressed data is output. The present invention generates a control signal to automatically adjust the frame rate, quantization table and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization.

[0096] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the image processing method, which includes: adjusting the parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameter includes the capture frame rate, the YUV data output format and the quantization table category; the first control signal is generated according to the monitoring result by monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result by monitoring the response time of the output write operation reply; the image data is obtained according to the current value of the capture frame rate; the image data is processed according to the current value of the YUV data output format to obtain the data processing result; the data processing result is compressed using the corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and the compressed data is output. In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing method provided by the above-mentioned methods, which includes: adjusting the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include a capture frame rate, a YUV data output format and a quantization table category; the first control signal is generated based on the monitoring result of monitoring the VGA write memory bus; the second control signal is generated based on the monitoring result of monitoring the response time of the output write operation reply; image data is obtained according to the current value of the capture frame rate; data processing is performed on the image data according to the current value of the YUV data output format to obtain a data processing result; the data processing result is compressed using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and the compressed data is output. On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to execute the image processing method provided by the above methods, the method comprising: adjusting the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include a capture frame rate, a YUV data output format and a quantization table category; the first control signal is generated according to the monitoring result of monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result of monitoring the response time of the output write operation reply; acquiring image data according to the current value of the capture frame rate; performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; compressing the data processing result using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and outputting the compressed data. The above-described device embodiments are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those skilled in the art may understand and implement the solution without creative work.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: include: The parameter value of at least one configuration parameter is adjusted according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameters include capture frame rate, YUV data output format and quantization table category; the first control signal is generated according to the monitoring result of monitoring the VGA write memory bus; the second control signal is generated according to the monitoring result of monitoring the response time of the output write operation reply; acquiring image data according to a current value of the capture frame rate; Performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; Compressing the data processing result using a corresponding quantization table according to a current value of the quantization table category to obtain compressed data; The compressed data is output.

2. The image processing method based on the baseboard management controller according to claim 1, characterized in that: Monitoring the VGA write memory bus and generating a first control signal according to the monitoring result, specifically including: When the VGA write memory bus is operated and the write address falls within the current memory range, generating a detection valid signal; When the number of the detection effective signals in one frame of image data is greater than a first preset number threshold, a detection effective output signal is generated as a first control signal.

3. The image processing method based on baseboard management controller according to claim 1, characterized in that: Monitoring the response time of the output write operation reply and generating a second control signal according to the monitoring result specifically includes: If the current output write operation is valid, record the first time it takes to receive the reply to the current write operation; When the first time is not less than a preset time threshold, a bandwidth shortage flag signal is output as a second control signal.

4. The image processing method based on baseboard management controller according to claim 2, characterized in that: Adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the detection effective output signal, calculating the image update frequency according to the number of the detection effective output signals acquired within a preset time range; In the case where the image update frequency is greater than a first preset update threshold, detecting a current capture frame rate; if the current capture frame rate is less than a first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; In the case where the image update frequency is not greater than a first preset update threshold and greater than a second preset update threshold, detecting a current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; When the image update frequency is not greater than a second preset update threshold, the current capture frame rate is determined according to the number of times the valid output signal is detected per second.

5. The image processing method based on baseboard management controller according to claim 3, characterized in that: Adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the insufficient bandwidth flag signal, recording a target number of receiving the insufficient bandwidth flag signal within a frame of image data; When the target quantity is not less than a second preset quantity threshold, detecting a current capture frame rate; If the current capture frame rate is greater than a third preset frame rate threshold, executing a first adjustment strategy; If the current capture frame rate is not greater than a fourth preset frame rate threshold, the current YUV data output format is detected and a second adjustment strategy is executed.

6. The image processing method based on baseboard management controller according to claim 5, characterized in that: The first adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until a second bandwidth shortage flag signal is obtained; In response to obtaining the second bandwidth shortage flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth shortage flag signal is obtained; In response to acquiring the third insufficient bandwidth flag signal, the current YUV data output format is detected and a second adjustment strategy is executed.

7. The image processing method based on a baseboard management controller according to claim 5 or 6, characterized in that: The second adjustment strategy includes: Recording the currently received insufficient bandwidth flag signal as the first insufficient bandwidth flag signal; If the current YUV data output format is the first preset data format, the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until a second bandwidth insufficient flag signal is obtained; In response to acquiring the second bandwidth insufficient flag signal, detecting the current quantization table and executing a third adjustment strategy; If the current YUV data output format is the second preset data format, detecting the current quantization table and executing a third adjustment strategy; Wherein, the third strategy includes: If the current quantization table is a high-quality quantization table, adjusting the current quantization table to a default quantization table; If the current quantization table is a default quantization table, the current quantization table is adjusted to a low-quality quantization table.

8. An image processing system, characterized in that: include: A monitoring unit, used for monitoring the VGA write memory bus and generating a first control signal according to the monitoring result; A detection unit, configured to generate the second control signal based on monitoring of a response time of a reply to an output write operation by a monitoring module and based on a monitoring result; A control unit, configured to adjust a parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain a current value of the at least one configuration parameter; wherein the configuration parameters include a capture frame rate, a YUV data output format, and a quantization table category; a data acquisition unit, configured to acquire image data according to a current value of the capture frame rate; A format processing unit, used for performing data processing on the image data according to the current value of the YUV data output format to obtain a data processing result; A compression unit, configured to compress the data processing result using a corresponding quantization table according to a current value of the quantization table category to obtain compressed data; An output unit is used to output the compressed data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image processing method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images