Image processing method and system

Abstract

The application provides an image processing method and system, comprising: adjusting parameter values of at least one configuration parameter according to a first control signal and a second control signal to obtain current values of the at least one configuration parameter; the first control signal is generated according to monitoring of a VGA write memory bus; the second control signal is generated according to monitoring of a response time of a write operation reply; obtaining image data according to the current values of a capture frame rate; performing data processing on the image data according to the current values of a YUV data output format to obtain a data processing result; compressing the data processing result by using a corresponding quantization table according to the current values of a quantization table category to obtain compressed data; and outputting the compressed data. The application generates control signals by detecting a VGA write memory bus and a response time of a write operation reply to adjust a frame rate, a quantization table and a YUV data output format, so that resource utilization is high.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and system. Background Technology

[0002] Baseboard Manager Controller (BMC) is commonly used in the server field. This chip and the system software running on it monitor the server's status (temperature, fan speed, main CPU operation, etc.). At the same time, it has an important function: to transmit local image information to a remote location via the network for remote display and monitoring, making it convenient for users to observe and operate remotely.

[0003] In existing technologies, when transmitting raw image data, if the data volume is enormous, it affects transmission efficiency. Therefore, images are typically compressed before transmission. In existing technologies, the image processing module acquires the raw image data according to the pre-defined configuration of the driver. Then, it performs data format conversion, YUV sampling, block conversion, and JPEGIP (DCT transform, quantization, and encoding) to compress the image data. Finally, the output module writes the processed compressed data to the DDR and outputs each frame's image information, including frame header, frame tail, and frame size. The driver reads this information and transmits it to the network, retrieves the correct image data from the DDR, and displays it remotely.

[0004] During this process, the image processing module can only perform image processing according to the configuration parameters set by the driver (including capture method, frame rate, YUV data output format, quantization table, etc.). This has the drawback of wasting DDR memory and DDR bus bandwidth. Especially when DDR bandwidth is insufficient, it will compete with other modules and affect each other. For example, if the image processing module is configured to acquire data through the AXI bus, with a frame rate of 60fp / s, a YUV data output format of YUV444, and the default quantization table, if insufficient bandwidth occurs, the image processing module will compete with the VGA (Video Graphics Array) module, resulting in VGA operation stuttering, screen distortion, and other phenomena. When users discover this situation, they can only modify the configuration parameters in the driver to reduce the image processing module's DDR usage, which is inefficient.

[0005] In summary, existing technologies suffer from low utilization of computing resources. Summary of the Invention

[0006] This invention provides an image processing method and system to address the shortcomings of low computing resource utilization in existing technologies and achieve image processing with high computing resource utilization.

[0007] This invention provides an image processing method, comprising: The parameter values ​​of at least one configuration parameter are adjusted according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; the first control signal is generated based on the monitoring result of the VGA write memory bus; the second control signal is generated based on the monitoring result of the monitoring of the response time of the output write operation reply. Image data is obtained based on the current value of the capture frame rate; The image data is processed according to the current value of the YUV data output format to obtain the data processing result; Based on the current value of the quantization table category, the data processing result is compressed using the corresponding quantization table to obtain compressed data; Output the compressed data.

[0008] According to an image processing method provided by the present invention, the VGA write memory bus is monitored, and a first control signal is generated based on the monitoring result, specifically including: When there is an operation on the VGA write memory bus and the write address falls within the current video memory range, a valid detection signal is generated; If the number of valid detection signals in a frame of image data is greater than a first preset threshold, a valid detection output signal is generated as a first control signal.

[0009] According to an image processing method provided by the present invention, the response time of the write operation reply of the output is monitored, and a second control signal is generated based on the monitoring result, specifically including: If the current write operation is valid, record the first time it takes to receive the reply to the current write operation; If the first time is not less than a preset time threshold, the output bandwidth insufficient flag signal is used as the second control signal.

[0010] According to an image processing method provided by the present invention, adjusting configuration parameters based on a first control signal and a second control signal specifically includes: In response to acquiring the valid detection output signal, the image update frequency is calculated based on the number of valid detection output signals acquired within a preset time range; If the image update frequency is greater than a first preset update threshold, the current capture frame rate is detected; if the current capture frame rate is less than the first preset frame rate threshold, the current capture frame rate is adjusted to the first frame rate. If the image update frequency is not greater than a first preset update threshold and is greater than a second preset update threshold, the current capture frame rate is detected; if the current capture frame rate is not less than the first preset frame rate threshold, the current capture frame rate is adjusted to the second frame rate. If the image update frequency is not greater than the second preset update threshold, the current capture frame rate is determined based on the number of times the valid output signal is detected per second.

[0011] According to an image processing method provided by the present invention, adjusting configuration parameters based on a first control signal and a second control signal specifically includes: In response to acquiring the bandwidth insufficiency flag signal, the number of targets that received the bandwidth insufficiency flag signal within a frame of image data is recorded. If the number of targets is not less than a second preset threshold, the current capture frame rate is detected. If the current capture frame rate is greater than the third preset frame rate threshold, then the first adjustment strategy is executed; If the current capture frame rate is not greater than the fourth preset frame rate threshold, then the current YUV data output format is detected and the second adjustment strategy is executed.

[0012] According to an image processing method provided by the present invention, the first adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until the second insufficient bandwidth flag signal is obtained; In response to the acquisition of the second insufficient bandwidth flag signal, the current capture frame rate is adjusted to the fourth preset frame rate threshold, and the adjusted capture frame rate is maintained until the third insufficient bandwidth flag signal is acquired. In response to the acquisition of the third insufficient bandwidth flag signal, the current YUV data output format is detected and the second adjustment strategy is executed.

[0013] According to an image processing method provided by the present invention, the second adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; If the current YUV data output format is the first preset data format, then the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until the second insufficient bandwidth flag signal is obtained. In response to acquiring the second insufficient bandwidth flag signal, the current quantization table is detected and a third adjustment strategy is executed; If the current YUV data output format is the second preset data format, then the current quantization table is detected and the third adjustment strategy is executed; The third strategy includes: If the current quantization table is a high-quality quantization table, then the current quantization table is adjusted to the default quantization table; If the current quantization table is the default quantization table, then the current quantization table will be adjusted to a low-quality quantization table.

[0014] The present invention also provides an image processing system, comprising: The monitoring unit is used to monitor the VGA write memory bus and generate the first control signal based on the monitoring result. The detection unit is used to generate the second control signal based on the monitoring result of the monitoring module monitoring the response time of the output write operation reply; A control unit is configured to adjust the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; A data acquisition unit is used to acquire image data based on the current value of the capture frame rate; The format processing unit is used to process the image data according to the current value of the YUV data output format to obtain the data processing result; A compression unit is used to compress the data processing result according to the current value of the quantization table category using the corresponding quantization table to obtain compressed data; The output unit is used to output the compressed data.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image processing method described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described above.

[0018] The image processing method and system provided by this invention adjust the parameter values ​​of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter. The configuration parameter includes a capture frame rate, a YUV data output format, and a quantization table type. The first control signal is generated based on the monitoring result of the VGA write memory bus. The second control signal is generated based on the monitoring result of the response time of the output write operation reply. Image data is acquired based on the current value of the capture frame rate. The image data is processed according to the current value of the YUV data output format to obtain a data processing result. The data processing result is compressed using the corresponding quantization table according to the current value of the quantization table type to obtain compressed data. The compressed data is then output. This invention automatically adjusts the frame rate, quantization table, and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts of the image processing method provided by the present invention.

[0021] Figure 2 This is a schematic diagram illustrating an application scenario of the image processing method provided by the present invention.

[0022] Figure 3 This is the second flowchart of the image processing method provided by the present invention.

[0023] Figure 4 This is the third flowchart of the image processing method provided by the present invention.

[0024] Figure 5 This is a schematic diagram of the image processing system provided by the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The following is combined with Figures 1-4 The image processing method of the present invention is described. Figure 1 This is one of the structural schematic diagrams of the image processing method provided by the present invention, such as... Figure 1 As shown, the method includes the following steps: This invention provides an image processing method, comprising the following steps: Step 110: Adjust the parameter value of at least one configuration parameter according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; the first control signal is generated based on the monitoring result of the VGA write memory bus; the second control signal is generated based on the monitoring result of the monitoring of the response time of the output write operation reply. Step 120: Obtain image data based on the current value of the capture frame rate; Step 130: Perform data processing on the image data according to the current value of the YUV data output format to obtain the data processing result; Step 140: Compress the data processing result using the corresponding quantization table based on the current value of the quantization table category to obtain compressed data; Step 150: Output the compressed data.

[0028] It should be explained that the applicable scenarios of this invention include using a BMC chip to transmit local image information via a network to a remote location for remote display and monitoring. It is important to emphasize that this invention does not impose specific limitations on the particular chip used. In one embodiment, the application scenario is as follows: Figure 2As shown, specifically, information from the host end is transmitted via PCIe to the VGA module inside the BMC chip. The VGA module interacts with the DDR, processes the data, and converts it into a DVI (Digital Visual Interface) signal for display on the local monitor. To display on a remote monitor, image data can be retrieved from the DDR (Double Data Rate) memory via the DVI interface or AXI (Advanced eXtensible Interface) bus. This image data is then compressed by the image processing module, written to the DDR, and finally transmitted over the network to the remote monitor, thus enabling remote management and control.

[0029] The AXI bus is an on-chip bus designed for high performance, high bandwidth, and low latency. It includes master devices, slave devices, and bus interconnect components, and has a multi-master, multi-slave structure.

[0030] Because existing image processing modules can only perform image processing according to fixed configuration parameters, there is a waste of computing resources, especially DDR memory and DDR bus bandwidth. This invention, in addition to the modules required to complete the necessary image processing flow—namely, a data acquisition module, a format processing module, a compression module, and an output module—adds a monitoring module, a control module, and a surveillance module.

[0031] The monitoring module is used to implement the monitoring function. By monitoring the signals on the VGA write memory bus (AXI write channel), it determines the frequency of the current memory content update and generates the first control signal to determine the frame rate at which the data acquisition module acquires data.

[0032] The monitoring module is used to monitor the response time of the DDR when the output module writes to the DDR. Based on the response time, it determines the bandwidth situation at this time and generates a second control signal to determine the frame rate of the data acquisition module and / or the YUV data output format of the format processing module and / or the quantization table type used by the compression module.

[0033] Based on the results from the listening and monitoring modules (i.e., the first and second control signals), the control module employs different strategies to reduce the time the image processing module occupies DDR memory. These strategies include adjusting the capture frame rate, the YUV data output format, and using quantization tables of different types (i.e., compression quality) to obtain the current values ​​of each configuration parameter. It is important to note that in each adjustment, the three configuration parameters are not necessarily adjusted simultaneously; the current value of a configuration parameter may be an updated value after adjustment or a historical value before adjustment.

[0034] Furthermore, a RAM module can be added. The RAM module stores quantization tables for different compression qualities, including three different quantization table categories: high quality, medium (default), and low quality. It is understood that higher compression quality results in more compressed image data, while lower compression quality results in less compressed image data. This invention uses RAM to store the quantization tables, avoiding the need to operate on a large number of registers when modifying the quantization tables, thus improving speed and simplifying operation.

[0035] The data acquisition module is used to acquire image data based on the current value of the capture frame rate; such as Figure 3 As shown, the listening module can be set within the data acquisition module.

[0036] The format processing module is used to process data according to the current value of the YUV data output format to obtain the data processing result. In some embodiments, the format processing module includes an RGB2YUV module and a YUV2BLOCK module, such as... Figure 3 As shown.

[0037] The compression module is used to compress the data processing results according to the current value of the quantization table category and the corresponding quantization table to obtain compressed data. In some embodiments, the compression module includes a JPEG IP module, such as... Figure 3 As shown.

[0038] The output module is used to output compressed data.

[0039] This invention enables automatic frame rate adjustment, avoids the drawbacks of traditional solutions, reduces DDR memory waste, improves DDR bus bandwidth utilization, optimizes overall chip performance, and enhances the user experience on the remote end.

[0040] The generation of the first control signal is further explained below. In some embodiments, the VGA write memory bus is monitored, and the first control signal is generated based on the monitoring result, specifically including: When there is an operation on the VGA write memory bus and the write address falls within the current video memory range, a valid detection signal is generated; If the number of valid detection signals in a frame of image data is greater than a first preset threshold, a valid detection output signal is generated as a first control signal.

[0041] Specifically, the monitoring module monitors the VGA write memory bus (i.e., the AXI write channel on the VGA). When it detects an operation on the AXI write channel on the VGA, it determines whether the write address falls within the current video memory range. If it does, it indicates that there is data update in the video memory.

[0042] It should be explained that the video memory range at this time is: video memory base address ~ video memory base address + address space size occupied by the current pixel; the address space size occupied by the current pixel is determined according to different bpp modes, and this invention does not limit it.

[0043] In one embodiment, the bpp mode is 32bpp, where one pixel value occupies 32 bits, or 4 bytes. Therefore, the address space occupied by the current pixel = number of pixels * 4.

[0044] In another embodiment, the bpp mode is 16bpp, where one pixel value occupies 16 bits, or 2 bytes. Therefore, the address space occupied by the current pixel = number of pixels * 2.

[0045] Furthermore, when an update is detected in the video memory, a valid detection signal is generated. Then, the number of valid detection signals within a frame of image data is recorded. If this number exceeds a first preset threshold, a valid detection output signal is triggered, and the first control signal becomes the valid detection output signal. Conversely, if the number is less than a preset threshold, a valid detection output signal is not triggered, and the first control signal is empty.

[0046] Furthermore, the present invention does not impose a specific limitation on the first preset quantity threshold. In some embodiments, the first preset quantity threshold is 1.

[0047] It should be noted that this invention does not limit the method of recording the number of valid signals detected within a frame of image data; the recording method can be selected according to the actual situation. In one embodiment, a counter cnt1 is set, and the counter value is incremented by 1 each time a valid write operation is detected.

[0048] Based on the above embodiments, when the number of write operations detected within a frame of image data (the value of cnt1) is greater than or equal to the set threshold 1, it is considered that the current frame of image data has changed, and a valid detection output signal is triggered; when the number of write operations detected within a frame of image data (the value of cnt1) is less than the set threshold 1, it is considered that the current frame of image data has not changed, and a valid detection output signal is not triggered.

[0049] The generation of the second control signal is further explained below. In some embodiments, the response time of the write operation reply of the output is monitored, and the second control signal is generated based on the monitoring result, specifically including: If the current write operation is valid, record the first time it takes to receive the reply to the current write operation; If the first time is not less than a preset time threshold, the output bandwidth insufficient flag signal is used as the second control signal.

[0050] Specifically, the monitoring module monitors the AXI write channel of the output module writing to DDR. When the output module initiates a valid write operation, that is, when awvalid and awready are both high, it records the first time taken to receive the reply of the current write operation.

[0051] Next, the first time interval is judged. If the first time interval is greater than or equal to a preset time threshold, it indicates that the bandwidth may be insufficient, and a bandwidth insufficient flag signal is output. In this case, the second control signal is the bandwidth insufficient flag signal. Conversely, if the first time interval is less than the preset time threshold, it indicates that the current DDR bandwidth is sufficient for the image processing module, and no adjustment is required. No flag signal is output, and the second control signal is empty.

[0052] Furthermore, the present invention does not impose a specific limitation on the preset time threshold. In some embodiments, the preset time threshold is 100 clock cycles.

[0053] It should be noted that this invention does not restrict the method of recording the first time taken to receive a response to the current write operation; the recording method can be selected according to the actual situation. In one embodiment, a counter cnt3 is set, and the counter value is incremented by 1 every clock cycle until a response to this write operation is received, i.e., when bvalid and bready are both high, the counting stops.

[0054] Based on the above embodiments, if the counter value cnt3 is less than the set threshold 3, it indicates that the bandwidth of DDR is sufficient for the image processing module to use, no adjustment is needed, and no flag signal is output; if the counter value cnt3 is greater than or equal to the set threshold 3, it indicates that the bandwidth may be insufficient, and a flag signal is output.

[0055] The following provides further explanation of the adjustment of configuration parameters. In some embodiments, adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: In response to acquiring the valid detection output signal, the image update frequency is calculated based on the number of valid detection output signals acquired within a preset time range; If the image update frequency is greater than a first preset update threshold, the current capture frame rate is detected; if the current capture frame rate is less than the first preset frame rate threshold, the current capture frame rate is adjusted to the first frame rate. If the image update frequency is not greater than a first preset update threshold and is greater than a second preset update threshold, the current capture frame rate is detected; if the current capture frame rate is not less than the first preset frame rate threshold, the current capture frame rate is adjusted to the second frame rate. If the image update frequency is not greater than the second preset update threshold, the current capture frame rate is determined based on the number of times the valid output signal is detected per second.

[0056] Specifically, the valid output signal is detected and transmitted to the control module, the number of valid output signals detected within a preset time range is recorded, and the image update frequency is calculated based on the preset time range and the number of valid output signals detected.

[0057] It should be noted that this invention does not limit the method of recording the number of valid output signals obtained within a preset time range; the recording method can be selected according to the actual situation. In one embodiment, a counter cnt2 is set, and the counter value is incremented by 1 each time a valid output signal is received from the monitoring module. The image update frequency is obtained by counting the image update frequency within 1 second. That is, in this embodiment, the preset time range is 1 second.

[0058] Then, the control module adjusts the current frame rate (i.e., capture frame rate) based on the image update frequency.

[0059] It should be noted that the present invention may also include a frame rate adjustment module, wherein the control module controls the frame rate adjustment module to adjust the capture frame rate according to the image update frequency. The present invention does not limit the specific adjustment method and module details.

[0060] During the adjustment process, if the image update frequency is greater than the first preset update threshold, it indicates that the image is updating relatively quickly, and at this time the current capture frame rate is detected.

[0061] This invention does not impose a specific limitation on the first preset update threshold. In one specific embodiment, the first preset update threshold is 30 image updates within 1 second.

[0062] The current capture frame rate is then determined. If the current capture frame rate is less than a first preset frame rate threshold, the current capture frame rate is adjusted to the first frame rate to achieve a better display effect. Conversely, if the current capture frame rate is not less than the first preset frame rate threshold, image data is captured according to the set frame rate, i.e., the current capture frame rate remains unchanged. It should be noted that the first preset frame rate threshold is less than the first frame rate, and this invention does not impose other restrictions on the first preset frame rate threshold or the first frame rate.

[0063] In one specific embodiment, the first preset frame rate threshold is 30 frames, and the first frame rate is 45 frames.

[0064] If the image update frequency is no greater than a first preset update threshold but greater than a second preset update threshold, it indicates that the image update is relatively slow. The current capture frame rate is then detected. This invention does not specifically limit the second preset update threshold. In one specific embodiment, the second preset update threshold is 10 image updates per second.

[0065] The current capture frame rate is then assessed. If it is not less than the first preset frame rate threshold, the current capture frame rate is adjusted to the second frame rate to reduce the transmission of invalid data and save bandwidth. Conversely, if the current capture frame rate is less than the first preset frame rate threshold, image data is captured according to the set frame rate, i.e., the current capture frame rate remains unchanged.

[0066] It should be noted that the second frame rate is less than the first preset frame rate, and the first preset frame rate is less than the first frame rate. This invention does not impose any other restrictions on the second frame rate, the first preset frame rate, and the first frame rate.

[0067] In one specific embodiment, the first frame rate is 45 frames, the first preset frame rate threshold is 30 frames, and the second frame rate is 20 frames.

[0068] If the image update frequency is no greater than a second preset update threshold, indicating slow image updates, the current capture mode is changed to monitoring mode. This means the monitoring module only starts working when it detects a valid output signal. Specifically, the current capture frame rate is determined based on the number of valid output signals detected per second. In other words, when the image update frequency is no greater than the second preset update threshold, monitoring mode is used, and the set capture frame rate has no effect. The current capture frame rate is determined by the number of valid output signals for capture. For example, if the image updates twice per second (two valid output signals), only two frames are captured per second. If the image updates seven times per second (seven valid output signals), only seven frames are captured per second. This invention obtains updated image data from DDR via the AXI bus, avoiding the transmission of invalid data and saving DDR memory and bandwidth.

[0069] Furthermore, in some embodiments, the adjustment of the configuration parameters further includes: adjusting the configuration parameters according to the first control signal and the second control signal, specifically including: In response to acquiring the bandwidth insufficiency flag signal, the number of targets that received the bandwidth insufficiency flag signal within a frame of image data is recorded. If the number of targets is not less than a second preset threshold, the current capture frame rate is detected. If the current capture frame rate is greater than the third preset frame rate threshold, then the first adjustment strategy is executed; If the current capture frame rate is not greater than the fourth preset frame rate threshold, then the current YUV data output format is detected and the second adjustment strategy is executed.

[0070] Specifically, the insufficient bandwidth flag signal is transmitted to the control module, which records the number of targets that receive the insufficient bandwidth flag signal within a frame of image data.

[0071] The control module adjusts the configuration parameters for this operation based on the target number to ensure that the relevant modules can work normally. If the target number is less than the second preset threshold, it means that the bandwidth is sufficient for the image processing module to operate in the current working mode and no adjustment is needed. If the target number is greater than or equal to the set second preset threshold, it means that the bandwidth is insufficient for the image processing module to operate in the current working mode and the corresponding parameters need to be adjusted. At this time, the current capture frame rate is detected.

[0072] It should be noted that this invention does not limit the method of recording the number of valid signals detected within a frame of image data; the recording method can be selected according to the actual situation. Based on the above embodiment, a counter cnt4 is set to count the number of times the cnt3 value exceeds the threshold 3 (i.e., a bandwidth insufficient flag signal is emitted) during the output of a frame of image data. It should be noted that after adjusting the adjustment parameters, the counter cnt4 value is reset to zero.

[0073] In one embodiment, the second preset quantity threshold is 100.

[0074] The following provides a further explanation of the adjustment strategy for the configuration parameters. During the adjustment process, the current capture frame rate is determined. If the current capture frame rate is greater than the third preset frame rate threshold, the first adjustment strategy is executed; if the current capture frame rate is less than or equal to the fourth preset frame rate threshold, the capture frame rate is not adjusted, the current YUV data output format is detected, and the second adjustment strategy is executed; if the current capture frame rate is neither greater than the third preset frame rate threshold nor less than the fourth preset frame rate threshold, the capture frame rate is adjusted to the fourth preset frame rate threshold.

[0075] It should be noted that the third preset frame rate threshold is greater than the fourth preset frame rate threshold, and the present invention does not impose any other restrictions on the third and fourth preset frame rate thresholds. In one specific embodiment, the third preset frame rate threshold is 30 frames per second, and the fourth preset frame rate threshold is 20 frames per second.

[0076] Furthermore, in some embodiments, the first adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until the second insufficient bandwidth flag signal is obtained; In response to the acquisition of the second insufficient bandwidth flag signal, the current capture frame rate is adjusted to the fourth preset frame rate threshold, and the adjusted capture frame rate is maintained until the third insufficient bandwidth flag signal is acquired. In response to the acquisition of the third insufficient bandwidth flag signal, the current YUV data output format is detected and the second adjustment strategy is executed.

[0077] Specifically, for ease of differentiation, the currently received bandwidth insufficiency flag signal is recorded as the first bandwidth insufficiency flag signal. Then, the current capture frame rate is judged. If the detected current capture frame rate is greater than a third preset frame rate threshold, the current capture frame rate is adjusted to the third preset frame rate threshold, and bandwidth monitoring continues. If the bandwidth is sufficient (i.e., no second bandwidth insufficiency flag signal is received), the current capture frame rate is maintained at the third preset frame rate threshold. If it is still insufficient (i.e., a second bandwidth insufficiency flag signal is received), the current capture frame rate is adjusted to a fourth preset frame rate threshold, and bandwidth monitoring continues. If the bandwidth is sufficient (i.e., no third bandwidth insufficiency flag signal is received), the current capture frame rate is maintained at the fourth preset frame rate threshold. If it is still insufficient (i.e., a second bandwidth insufficiency flag signal is received), a second adjustment strategy is executed.

[0078] Based on the above embodiments, the third preset frame rate threshold is 30 frames per second, and the fourth preset frame rate threshold is 20 frames per second. Therefore, the first strategy includes: if the frame rate is greater than 30, the data acquisition frame rate is adjusted to 30 frames per second, and bandwidth is monitored. If the bandwidth is sufficient, it remains at 30 frames per second. If it is still insufficient, the data acquisition frame rate is adjusted to 20 frames per second, and bandwidth is monitored. If the bandwidth is sufficient, it remains at 20 frames per second. If it is still insufficient, the second adjustment strategy is executed.

[0079] Furthermore, in some embodiments, the second adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; If the current YUV data output format is the first preset data format, then the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until the second insufficient bandwidth flag signal is obtained. In response to acquiring the second insufficient bandwidth flag signal, the current quantization table is detected and a third adjustment strategy is executed; If the current YUV data output format is the second preset data format, then the current quantization table is detected and the third adjustment strategy is executed; The third strategy includes: If the current quantization table is a high-quality quantization table, then the current quantization table is adjusted to the default quantization table; If the current quantization table is the default quantization table, then the current quantization table will be adjusted to a low-quality quantization table.

[0080] Specifically, for ease of differentiation, the currently received bandwidth insufficiency flag signal is recorded as the first bandwidth insufficiency flag signal. Then, the current YUV data output format is determined. If the current YUV data output format is a first preset data format, it is adjusted to a second preset data format, and bandwidth monitoring continues. If the bandwidth is sufficient (i.e., no second bandwidth insufficiency flag signal is received), the YUV data output format remains the second preset data format. If it is still insufficient (i.e., a second bandwidth insufficiency flag signal is received), the current quantization table is checked, and a third strategy is executed. Conversely, if the current YUV data output format is the second preset data format, no adjustment is made, the current quantization table is checked, and the third strategy is executed.

[0081] In one specific embodiment, the first preset data format is YUV444, and the second preset data format is YUV420.

[0082] The third strategy includes: if the current quantization table is a high-quality quantization table, it can be adjusted to a default (medium) quantization table; if the current quantization table is a default quantization table, it can be adjusted to a low-quality quantization table; if the current quantization table is a low-quality quantization table, no adjustment is made.

[0083] This invention employs adaptive adjustment of frame rate, YUV data output format, and quantization table to avoid the transmission of invalid data, save DDR memory, improve DDR bus bandwidth utilization, and reduce bandwidth requirements.

[0084] Furthermore, users can configure different thresholds according to their needs to control the display effect for the multiple thresholds mentioned in this invention.

[0085] Based on the above embodiments, in one embodiment, the internal structure of the control module is as follows: Figure 4 As shown.

[0086] The counting module 1 counts the valid signals transmitted from the monitoring module in 1-second increments, and then transmits the statistical values ​​to the comparison module 1. The comparison module 1 compares the statistical count with the data acquisition frame rate. Based on the comparison results, the frame rate of the data acquisition module is adjusted by the frame rate adjustment module to save DDR memory and bandwidth. The insufficient bandwidth flag signal transmitted from the counting module 2 and the statistical monitoring module is processed in one frame as the time unit. The statistical value is then transmitted to the comparison module 2. The comparison module 2 compares the statistical value with the current data acquisition frame rate. Based on the comparison results, the frame rate adjustment module adjusts the frame rate of the data acquisition module to save DDR memory and bandwidth. If the bandwidth is still insufficient after adjusting the frame rate, the strategy of adjusting the YUV data output format is used (second strategy). If the bandwidth is still insufficient after adjusting the YUV data output format, the strategy of adjusting the quantization table is used (third strategy).

[0087] The image processing method provided by this invention adjusts the parameter values ​​of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter. The configuration parameters include a capture frame rate, a YUV data output format, and a quantization table type. The first control signal is generated based on the monitoring result of the VGA write memory bus. The second control signal is generated based on the monitoring result of the response time of the output write operation reply. Image data is acquired based on the current value of the capture frame rate. The image data is processed according to the current value of the YUV data output format to obtain a data processing result. The data processing result is compressed using the corresponding quantization table according to the current value of the quantization table type to obtain compressed data. The compressed data is then output. This invention automatically adjusts the frame rate, quantization table, and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization.

[0088] The image processing system provided by the present invention is described below. The image processing system described below can be referred to in correspondence with the image processing method described above. Figure 5 This is a schematic diagram of the image processing system provided by the present invention, as shown below. Figure 5 As shown, the system includes the following modules: The monitoring unit 510 is used to monitor the VGA write memory bus and generate a first control signal based on the monitoring result. The detection unit 520 is used to generate the second control signal based on the monitoring result of the monitoring module monitoring the response time of the output write operation reply; The control unit 530 is configured to adjust the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; The data acquisition unit 540 is used to acquire image data based on the current value of the capture frame rate; The format processing unit 550 is used to process the image data according to the current value of the YUV data output format to obtain the data processing result; Compression unit 560 is used to compress the data processing result according to the current value of the quantization table category using the corresponding quantization table to obtain compressed data; Output unit 570 is used to output the compressed data.

[0089] According to an image processing system provided by the present invention, the system monitors the VGA write memory bus and generates a first control signal based on the monitoring result, specifically including: When there is an operation on the VGA write memory bus and the write address falls within the current video memory range, a valid detection signal is generated; If the number of valid detection signals in a frame of image data is greater than a first preset threshold, a valid detection output signal is generated as a first control signal.

[0090] According to an image processing system provided by the present invention, the response time of the write operation reply of the output is monitored, and a second control signal is generated based on the monitoring result, specifically including: If the current write operation is valid, record the first time it takes to receive the reply to the current write operation; If the first time is not less than a preset time threshold, the output bandwidth insufficient flag signal is used as the second control signal.

[0091] According to an image processing system provided by the present invention, adjusting configuration parameters based on a first control signal and a second control signal specifically includes: In response to acquiring the valid detection output signal, the image update frequency is calculated based on the number of valid detection output signals acquired within a preset time range; If the image update frequency is greater than a first preset update threshold, the current capture frame rate is detected; if the current capture frame rate is less than the first preset frame rate threshold, the current capture frame rate is adjusted to the first frame rate. If the image update frequency is not greater than a first preset update threshold and is greater than a second preset update threshold, the current capture frame rate is detected; if the current capture frame rate is not less than the first preset frame rate threshold, the current capture frame rate is adjusted to the second frame rate. If the image update frequency is not greater than the second preset update threshold, the current capture frame rate is determined based on the number of times the valid output signal is detected per second.

[0092] According to an image processing system provided by the present invention, adjusting configuration parameters based on a first control signal and a second control signal specifically includes: In response to acquiring the bandwidth insufficiency flag signal, the number of targets that received the bandwidth insufficiency flag signal within a frame of image data is recorded. If the number of targets is not less than a second preset threshold, the current capture frame rate is detected. If the current capture frame rate is greater than the third preset frame rate threshold, then the first adjustment strategy is executed; If the current capture frame rate is not greater than the fourth preset frame rate threshold, then the current YUV data output format is detected and the second adjustment strategy is executed.

[0093] According to an image processing system provided by the present invention, the first adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; Adjust the current capture frame rate to the third preset frame rate threshold, and maintain the adjusted capture frame rate until the second insufficient bandwidth flag signal is obtained; In response to the acquisition of the second insufficient bandwidth flag signal, the current capture frame rate is adjusted to the fourth preset frame rate threshold, and the adjusted capture frame rate is maintained until the third insufficient bandwidth flag signal is acquired. In response to the acquisition of the third insufficient bandwidth flag signal, the current YUV data output format is detected and the second adjustment strategy is executed.

[0094] According to an image processing system provided by the present invention, the second adjustment strategy includes: Record the currently received bandwidth insufficiency flag signal as the first bandwidth insufficiency flag signal; If the current YUV data output format is the first preset data format, then the current YUV data output format is adjusted to the second preset data format, and the adjusted data format is maintained until the second insufficient bandwidth flag signal is obtained. In response to acquiring the second insufficient bandwidth flag signal, the current quantization table is detected and a third adjustment strategy is executed; If the current YUV data output format is the second preset data format, then the current quantization table is detected and the third adjustment strategy is executed; The third strategy includes: If the current quantization table is a high-quality quantization table, then the current quantization table is adjusted to the default quantization table; If the current quantization table is the default quantization table, then the current quantization table will be adjusted to a low-quality quantization table.

[0095] The image processing system provided by this invention adjusts the parameter values ​​of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter. The configuration parameter includes a capture frame rate, a YUV data output format, and a quantization table type. The first control signal is generated based on the monitoring result of the VGA write memory bus. The second control signal is generated based on the monitoring result of the response time of the output write operation reply. Image data is acquired based on the current value of the capture frame rate. Data processing is performed on the image data according to the current value of the YUV data output format to obtain a data processing result. The data processing result is compressed using the corresponding quantization table according to the current value of the quantization table type to obtain compressed data. The compressed data is then output. This invention automatically adjusts the frame rate, quantization table, and YUV data output format by detecting the VGA write memory bus and the response time of the output write operation reply, thereby reducing the transmission of a large amount of invalid data and achieving high resource utilization.

[0096] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an image processing method, which includes: adjusting the parameter values ​​of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameter includes a capture frame rate, a YUV data output format, and a quantization table category; the first control signal is generated based on the monitoring result of listening to the VGA write memory bus; the second control signal is generated based on the monitoring result of monitoring the response time of the output write operation reply; acquiring image data according to the current value of the capture frame rate; processing the image data according to the current value of the YUV data output format to obtain a data processing result; compressing the data processing result using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and outputting the compressed data. Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the image processing method provided by the above-described methods. The method includes: adjusting the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein the configuration parameter includes a capture frame rate, a YUV data output format, and a quantization table category; the first control signal is generated based on the monitoring result of listening to the VGA write memory bus; the second control signal is generated based on the monitoring result of monitoring the response time of the output write operation reply; acquiring image data according to the current value of the capture frame rate; processing the image data according to the current value of the YUV data output format to obtain a data processing result; compressing the data processing result using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and outputting the compressed data. Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image processing method provided by the methods described above. The method includes: adjusting the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain a current value of at least one configuration parameter; wherein the configuration parameter includes a capture frame rate, a YUV data output format, and a quantization table category; the first control signal is generated based on a monitoring result of a VGA write memory bus; the second control signal is generated based on a monitoring result of a response time to an output write operation reply; acquiring image data according to the current value of the capture frame rate; processing the image data according to the current value of the YUV data output format to obtain a data processing result; compressing the data processing result using a corresponding quantization table according to the current value of the quantization table category to obtain compressed data; and outputting the compressed data. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that, include: The parameter values ​​of at least one configuration parameter are adjusted according to the first control signal and the second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; the first control signal is generated based on the monitoring result of the VGA write memory bus; the second control signal is generated based on the monitoring result of the monitoring of the response time of the output write operation reply. Image data is obtained based on the current value of the capture frame rate; The image data is processed according to the current value of the YUV data output format to obtain the data processing result; Based on the current value of the quantization table category, the data processing result is compressed using the corresponding quantization table to obtain compressed data; Output the compressed data; The VGA write memory bus is monitored, and a first control signal is generated based on the monitoring result. Specifically, this includes: generating a valid detection signal when there is an operation on the VGA write memory bus and the write address falls within the current memory range; and generating a valid detection output signal as the first control signal when the number of valid detection signals in a frame of image data is greater than a first preset threshold. Adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: in response to acquiring the detected valid output signal, calculating the image update frequency based on the number of detected valid output signals acquired within a preset time range; when the image update frequency is greater than a first preset update threshold, detecting the current capture frame rate; if the current capture frame rate is less than the first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; when the image update frequency is not greater than the first preset update threshold but greater than a second preset update threshold, detecting the current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; when the image update frequency is not greater than the second preset update threshold, determining the current capture frame rate based on the number of times the valid output signal is detected per second; In response to acquiring a bandwidth insufficiency flag signal, the number of targets that received the bandwidth insufficiency flag signal within a frame of image data is recorded; If the number of targets is not less than a second preset threshold, the current capture frame rate is detected. If the current capture frame rate is greater than the third preset frame rate threshold, then the first adjustment strategy is executed; If the current capture frame rate is not greater than the fourth preset frame rate threshold, then the current YUV data output format is detected and the second adjustment strategy is executed; The first adjustment strategy includes: recording the currently received bandwidth insufficiency flag signal as a first bandwidth insufficiency flag signal; adjusting the current capture frame rate to the third preset frame rate threshold, and maintaining the adjusted capture frame rate until a second bandwidth insufficiency flag signal is obtained; in response to obtaining the second bandwidth insufficiency flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth insufficiency flag signal is obtained; in response to obtaining the third bandwidth insufficiency flag signal, detecting the current YUV data output format and executing a second adjustment strategy. The second adjustment strategy includes: recording the currently received bandwidth shortage flag signal as the first bandwidth shortage flag signal; if the current YUV data output format is a first preset data format, then adjusting the current YUV data output format to a second preset data format, and maintaining the adjusted data format until the second bandwidth shortage flag signal is obtained.

2. The image processing method according to claim 1, characterized in that, The response time of the write operation reply to the output is monitored, and a second control signal is generated based on the monitoring result, specifically including: If the current write operation is valid, record the first time it takes to receive the reply to the current write operation; If the first time is not less than a preset time threshold, the output bandwidth insufficient flag signal is used as the second control signal.

3. The image processing method according to claim 1, characterized in that, The second adjustment strategy includes: In response to acquiring the second insufficient bandwidth flag signal, the current quantization table is detected and a third adjustment strategy is executed; If the current YUV data output format is the second preset data format, then the current quantization table is detected and the third adjustment strategy is executed; The third adjustment strategy includes: If the current quantization table is a high-quality quantization table, then the current quantization table is adjusted to the default quantization table; If the current quantization table is the default quantization table, then the current quantization table will be adjusted to a low-quality quantization table.

4. An image processing system, characterized in that, include: The monitoring unit is used to monitor the VGA write memory bus and generate the first control signal based on the monitoring result. The VGA write memory bus is monitored, and a first control signal is generated based on the monitoring result. Specifically, this includes: generating a valid detection signal when there is an operation on the VGA write memory bus and the write address falls within the current memory range; and generating a valid detection output signal as the first control signal when the number of valid detection signals in a frame of image data is greater than a first preset threshold. The detection unit is used to generate the second control signal based on the monitoring module's response time to the write operation reply. A control unit is configured to adjust the parameter value of at least one configuration parameter according to a first control signal and a second control signal to obtain the current value of at least one configuration parameter; wherein, the configuration parameter includes capture frame rate, YUV data output format and quantization table type; Adjusting the configuration parameters according to the first control signal and the second control signal specifically includes: in response to acquiring the detected valid output signal, calculating the image update frequency based on the number of detected valid output signals acquired within a preset time range; when the image update frequency is greater than a first preset update threshold, detecting the current capture frame rate; if the current capture frame rate is less than the first preset frame rate threshold, adjusting the current capture frame rate to a first frame rate; when the image update frequency is not greater than the first preset update threshold but greater than a second preset update threshold, detecting the current capture frame rate; if the current capture frame rate is not less than the first preset frame rate threshold, adjusting the current capture frame rate to a second frame rate; when the image update frequency is not greater than the second preset update threshold, determining the current capture frame rate based on the number of times the valid output signal is detected per second; In response to acquiring a bandwidth insufficiency flag signal, the number of targets that received the bandwidth insufficiency flag signal within a frame of image data is recorded; If the number of targets is not less than a second preset threshold, the current capture frame rate is detected. If the current capture frame rate is greater than the third preset frame rate threshold, then the first adjustment strategy is executed; If the current capture frame rate is not greater than the fourth preset frame rate threshold, then the current YUV data output format is detected and the second adjustment strategy is executed; The first adjustment strategy includes: recording the currently received bandwidth insufficiency flag signal as a first bandwidth insufficiency flag signal; adjusting the current capture frame rate to the third preset frame rate threshold, and maintaining the adjusted capture frame rate until a second bandwidth insufficiency flag signal is obtained; in response to obtaining the second bandwidth insufficiency flag signal, adjusting the current capture frame rate to a fourth preset frame rate threshold, and maintaining the adjusted capture frame rate until a third bandwidth insufficiency flag signal is obtained; in response to obtaining the third bandwidth insufficiency flag signal, detecting the current YUV data output format and executing a second adjustment strategy. The second adjustment strategy includes: recording the currently received bandwidth shortage flag signal as the first bandwidth shortage flag signal; if the current YUV data output format is a first preset data format, then adjusting the current YUV data output format to a second preset data format, and maintaining the adjusted data format until the second bandwidth shortage flag signal is obtained; A data acquisition unit is used to acquire image data based on the current value of the capture frame rate; The format processing unit is used to process the image data according to the current value of the YUV data output format to obtain the data processing result; A compression unit is used to compress the data processing result according to the current value of the quantization table category using the corresponding quantization table to obtain compressed data; The output unit is used to output the compressed data.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image processing method as described in any one of claims 1-3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1-3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1-3.

Images