Embedded firmware camera data modification methods, devices, media and equipment

Abstract

This invention discloses an embedded firmware camera data modification method, apparatus, medium, and device, relating to the field of computer vision technology. The method includes: real-time acquisition of image data and preliminary processing; using frame difference and background modeling algorithms to divide the pre-processed image data into regions and assign priorities, generating a region priority mapping table; extracting feature points of high-priority motion regions in the optimized image data using optical flow, and calculating motion vectors between consecutive frames; triggering an event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames, automatically adjusting the frame rate and processing accuracy; and using the motion vectors extracted by optical flow to calculate the object's motion speed in real time, dynamically adjusting the frame rate and processing accuracy according to the motion speed, ensuring sufficient detail is captured during rapid motion while saving processing resources when stationary.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an embedded firmware method, apparatus, medium, and device for modifying camera data. Background Technology

[0002] With the rapid development of computer vision and image processing technologies, the data processing requirements of cameras in embedded systems are becoming increasingly complex. Traditional embedded cameras are mainly used for image acquisition and transmission in static scenes, with little involvement in real-time dynamic scene processing. However, with the expansion of fields such as the Internet of Things, intelligent monitoring, and autonomous driving, the demand for data processing in real-time dynamic scenes is becoming increasingly urgent. Existing embedded camera systems lack flexible frame rate adjustment and region priority processing mechanisms, and are often unable to efficiently process large amounts of dynamic scene data under bandwidth-constrained conditions. At the same time, existing technologies mostly use image compression methods with fixed compression ratios and processing precision, which cannot be flexibly adjusted according to scene changes, resulting in low data transmission efficiency and loss of important information.

[0003] The shortcomings of existing technologies are mainly reflected in two aspects: First, existing embedded camera systems lack differentiated processing mechanisms for different regions when handling dynamic scenes, adopting a uniform processing strategy for the entire image data, which fails to effectively distinguish important moving areas from static background areas, leading to wasted resources; Second, existing systems cannot dynamically adjust the frame rate and processing precision according to the speed of moving objects, resulting in insufficient detail capture in fast-moving scenes, while producing excessive redundant data in static scenes. Therefore, there is an urgent need for an embedded camera system that can dynamically adjust the frame rate, process image data in layers, and perform efficient compression. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an embedded firmware camera data modification method to solve the problems of lack of differentiated processing for dynamic scenes and inability to adaptively adjust frame rate and processing accuracy in existing embedded camera systems.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an embedded firmware method for modifying camera data, comprising,

[0008] Real-time acquisition of image data and preliminary processing;

[0009] The frame difference method and background modeling algorithm are used to divide the pre-processed image data into regions and assign priorities to generate a region priority mapping table.

[0010] Based on the region priority mapping table, the image data is processed in layers to obtain optimized image data and feature information;

[0011] Feature points of high-priority motion regions in optimized image data are extracted using optical flow method, and motion vectors between consecutive frames are calculated.

[0012] Based on the calculated motion vectors between consecutive frames, an event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing precision.

[0013] Based on the adjusted frame rate and processing precision, the image data is compressed, and the compressed image data is output.

[0014] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the preliminary processing includes noise reduction, histogram equalization, color interpolation and gamma correction processing.

[0015] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the method involves dividing the pre-processed image data into regions and assigning priorities using frame difference and background modeling algorithms to generate a region priority mapping table, comprising the following steps.

[0016] After initial processing, the image data is stored in the frame buffer, forming image frame data;

[0017] Read the image data of the current frame and the previous frame from the image frame data, and calculate the pixel difference between the current frame and the previous frame for each pixel;

[0018] The calculated pixel differences are binarized to form a preliminary binarized image of the moving area and the static background area;

[0019] A background modeling algorithm is introduced to distinguish the background state of each pixel in a binary image at different time periods and construct a background model.

[0020] The pixels in the current frame are matched with the background model to determine whether the pixels in the current frame belong to the background, and the moving area and static background area are further divided.

[0021] Based on the division results of the motion area and the static background area, different areas are assigned priorities;

[0022] Based on the regional priority allocation results, a regional priority mapping table is generated.

[0023] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the following steps are included: performing layered processing on image data according to a region priority mapping table to obtain optimized image data and feature information.

[0024] Obtain the generated region priority mapping table and the corresponding image frames, and divide the image into high-priority motion regions and low-priority static background regions;

[0025] For high-priority motion regions, the Canny edge detection algorithm is used to extract the contours of moving objects and perform edge classification to generate the final edge map;

[0026] Based on the generated final edge map, the Harris corner detection method is used to extract corner features from the image;

[0027] For low-priority static background regions, a downsampling process is used to divide the image of the static background region into blocks for downsampling, resulting in a downsampled image.

[0028] The results of high-priority motion region processing are combined with low-priority downsampled background regions to form a complete image frame;

[0029] The corner features of high-priority moving regions and the basic features of low-priority static background regions are combined to form complete feature data.

[0030] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the following steps are included: extracting feature points of high-priority motion regions in the optimized image data using optical flow, and calculating motion vectors between consecutive frames.

[0031] Extract strong corner points in the motion region from the complete feature data as key feature points;

[0032] Based on optical flow, the brightness of a key feature point in an image is defined to remain constant across consecutive frames.

[0033] The optical flow vector is solved within a neighborhood window of the key feature points using the least squares method to obtain the motion vector of each feature point, expressed as:

[0034]

[0035] Where u represents the motion vector component of the feature point in the x-axis direction, v represents the motion vector component of the feature point in the y-axis direction, m represents the feature point, and I x I represents the brightness gradient of the image along the x-axis. y I represents the brightness gradient of the image along the y-axis. t This represents the change in image brightness over time t;

[0036] Based on the motion vector of each feature point, the feature points of the previous frame are matched with the feature points of the current frame to find the position of the same feature point in consecutive frames.

[0037] The complete motion trajectory of each feature point is generated by accumulating the optical flow vector of each feature point.

[0038] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the following steps are included: triggering an event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames to automatically adjust the frame rate and processing accuracy.

[0039] Based on the complete motion trajectory of the feature points, determine the displacement of the feature points between consecutive frames of the image;

[0040] Determine the velocity of a moving object based on the displacement between consecutive frames;

[0041] Based on the speed of moving objects, an event-driven frame processing mechanism is triggered to dynamically adjust the frame rate and processing precision.

[0042] Set a speed threshold; when the speed of a moving object exceeds the speed threshold, increase the frame rate and improve image processing accuracy.

[0043] When the speed of a moving object is less than a speed threshold, the frame rate is reduced and the image processing precision is simplified.

[0044] As a preferred embodiment of the embedded firmware camera data modification method of the present invention, the following steps are included: compressing the image data based on the adjusted frame rate and processing precision, and outputting the compressed image data.

[0045] For high-priority areas that have undergone frame rate adjustment and precision processing, a low compression ratio is used for compression.

[0046] For low-priority areas that have undergone frame rate adjustment and precision processing, high compression ratio is used for compression;

[0047] The compression results of high-priority and low-priority regions are merged to generate the compressed complete image data;

[0048] The motion vector and trajectory data of each feature point are encapsulated and merged with the compressed complete image data to obtain a compressed complete image data packet.

[0049] Secondly, the present invention provides an embedded firmware camera data modification device, comprising,

[0050] The acquisition and processing module acquires image data in real time and performs preliminary processing.

[0051] The partitioning module uses frame difference method and background modeling algorithm to divide the pre-processed image data into regions and assign priorities, generating a region priority mapping table;

[0052] The layering module performs layered processing on the image data according to the region priority mapping table to obtain optimized image data and feature information;

[0053] The extraction module extracts feature points of high-priority motion regions in the optimized image data using optical flow and calculates motion vectors between consecutive frames.

[0054] The frame processing module triggers an event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames, automatically adjusting the frame rate and processing precision.

[0055] The compression module compresses the image data based on the adjusted frame rate and processing precision, and outputs the compressed image data.

[0056] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the embedded firmware camera data modification method as described in the first aspect of the present invention.

[0057] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the embedded firmware camera data modification method as described in the first aspect of the present invention.

[0058] The beneficial effects of this invention are as follows: It achieves precise division of moving and static regions through frame difference and background modeling algorithms, and assigns priority to each region based on its importance, ensuring that the system can prioritize processing high-priority moving regions; it employs high-precision edge detection and corner extraction techniques for high-priority regions, while downsampling is performed on low-priority regions, effectively reducing unnecessary computational overhead and data redundancy through layered processing; the motion vector extracted by optical flow can calculate the object's speed in real time, and dynamically adjust the frame rate and processing accuracy according to the speed, ensuring sufficient detail is captured during rapid motion while saving processing resources during static conditions; and it uses differentiated compression ratios based on region priority for image compression, ensuring image quality in important regions while reducing storage and transmission overhead in low-priority regions, thus achieving efficient data processing and transmission under limited bandwidth conditions. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of the embedded firmware camera data modification method in Example 1.

[0061] Figure 2 This is a schematic diagram of the embedded firmware camera data modification system in Example 1. Detailed Implementation

[0062] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0063] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0064] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0065] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides an embedded firmware camera data modification method, including the following steps:

[0066] S1. Real-time acquisition of image data and preliminary processing.

[0067] S1.1 Preliminary processing includes denoising, histogram equalization, color interpolation, and gamma correction.

[0068] Specifically, denoising refers to smoothing brightness or color variations in an image and removing small random fluctuations.

[0069] After denoising, histogram equalization is performed on each frame to enhance image contrast. The steps are as follows:

[0070] For a grayscale image, the frequency of each grayscale level is counted to obtain a grayscale histogram, expressed as:

[0071]

[0072] Where J(i) represents the frequency of grayscale value i in the image, x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, M represents the width of the image, N represents the height of the image, i represents the grayscale value, and δ(I gray (x,y)-i) represents the Kronecker function, I gray (x,y) represents the gray value at pixel coordinates (x,y);

[0073] Furthermore, the expression for the Kronecker function is:

[0074]

[0075] If I gray (x,y)=I, the function value is 1, indicating that the gray value at pixel coordinates (x,y) belongs to gray value i. If I gray (x,y)≠i, the function value is 0, which means that the gray value at pixel coordinates (x,y) does not belong to the gray value i;

[0076] By using the cumulative distribution function, the probability of each gray value is summed to obtain the cumulative probability, expressed as:

[0077]

[0078] Where CDF(i) represents the cumulative probability of gray value i, j represents the gray value index, and H(j) represents the frequency of the j-th gray value in the image;

[0079] Based on the cumulative probability, the grayscale value of each pixel in the original image is remapped to a new grayscale value, expressed as:

[0080]

[0081] Among them, I' gray (x,y) represents the grayscale value remapped at pixel coordinates (x,y), and CDF(I(x,y)) represents the grayscale value I at pixel coordinates (x,y). gray The cumulative probability of (x,y), CDF min Let E represent the minimum cumulative probability, E represent the total number of pixels in the image, and L represent the number of gray levels.

[0082] Color interpolation refers to the use of the classic trilinear interpolation algorithm to convert monochrome pixels into a complete three-channel color image. The expression is:

[0083]

[0084] Where G represents green, R represents red, B represents blue, and H represents green. RGB(x,y) represents the RGB three-channel values ​​at pixel coordinates (x,y), H RGB (x-1,Y) represents the RGB three-channel values ​​at the neighboring pixel coordinates (x-1,y), H RGB (x+1,y) represents the RGB three-channel values ​​at the neighboring pixel coordinates (x+1,Y), H RGB (x,Y-1) represents the RGB three-channel values ​​at the neighboring pixel coordinates (X,Y-1), H RGB (x,y+1) represents the RGB three-channel values ​​at the nearest pixel coordinates (x,Y+1);

[0085] After color interpolation, the image brightness remains linear, making it unsuitable for direct display or further processing. To make the image brightness more consistent with the non-linear response of the human visual system, gamma correction is needed. The expression is:

[0086]

[0087] Among them, L cor (x,y) represents the color value at pixel coordinates (x,y) after gamma correction. γ represents the gamma correction parameter, which indicates the degree of non-linearity in the human eye's perception of brightness. A common γ value is 2.2, which means that the human eye is more sensitive to darker areas than to brighter areas.

[0088] S2. The pre-processed image data is divided into regions and assigned priorities using the frame difference method and background modeling algorithm, generating a region priority mapping table.

[0089] S2.1 The image data after preliminary processing is stored in the frame buffer to form image frame data; read the image data of the current frame and the previous frame in the image frame data, and calculate the pixel difference between the current frame and the previous frame for each pixel.

[0090] Specifically, the expression for the pixel difference is:

[0091] D(x,y)=|Q t (x,y)-Q t-1 (x,y)|;

[0092] Where D(x,y) represents the pixel difference at pixel coordinates (x,y), a larger difference indicates that the pixel may belong to a moving region, and a smaller difference indicates that the pixel may belong to a static background region, Q t (x,y) represents the pixel value at pixel coordinates (x,y) at time t. This pixel value is either a grayscale value or an RGB three-channel value. Q t-1 (x,y) represents the pixel value at pixel coordinates (x,y) at time t-1.

[0093] S2.2. The calculated pixel differences are binarized to form a preliminary binarized image of the moving area and the static background area.

[0094] Specifically, the expression for binarization is:

[0095]

[0096] Where P(x,y) represents the binary result at pixel coordinates (x,y), 1 indicates that the pixel belongs to the moving region, 0 indicates that the pixel belongs to the static background region, and T represents the threshold for distinguishing between the moving region and the static background region. When the pixel difference D(x,y) exceeds the threshold T, the pixel is considered to be moving; otherwise, the pixel is considered to be stationary.

[0097] S2.3 Introduce a background modeling algorithm to distinguish the background state of each pixel position in the binary image at different time periods and construct a background model; match the pixels in the current frame with the background model to determine whether the pixels in the current frame belong to the background, and further divide the moving area and the static background area.

[0098] Specifically, the background model is usually composed of multiple Gaussian distributions. A common practice is to use a Gaussian Mixture Model (GMM) to model the background of each pixel. If the pixel value in the current frame matches a Gaussian distribution in the background model (i.e., the pixel value is close enough to the mean of the Gaussian distribution), then the mean and variance of the Gaussian distribution need to be updated.

[0099] Furthermore, the mean update expression for the background model is:

[0100]

[0101] in, Indicates the updated number k The mean of a Gaussian distribution, α This indicates the update rate of the background model. Indicates the number before the update k The mean of a Gaussian distribution, k Indicates a Gaussian distribution index;

[0102] The variance update expression for the background model is:

[0103]

[0104] in, Indicates the updated number k The variance of a Gaussian distribution Indicates the number before the update kThe variance of a Gaussian distribution;

[0105] Based on the updated results of the background model, it is determined whether the current pixel belongs to the background. If a pixel does not conform to any Gaussian distribution, it is considered to belong to the foreground region (i.e., the motion region); otherwise, it is considered to belong to the background region.

[0106] S2.4. Based on the division results of the motion area and the static background area, assign priorities to different areas; based on the area priority allocation results, generate an area priority mapping table.

[0107] Specifically, the allocation of priorities follows these rules:

[0108] For detected motion regions, higher priority is assigned based on relative importance; for static background regions, lower priority is assigned.

[0109] S3. Based on the region priority mapping table, the image data is processed in layers to obtain optimized image data and feature information.

[0110] S 3.1. Obtain the generated region priority mapping table and the corresponding image frames, and divide the image into high-priority motion regions and low-priority static background regions; for high-priority motion regions, use the Canny edge detection algorithm to extract the contours of moving objects and perform edge classification to generate the final edge map.

[0111] Specifically, the Canny edge detection algorithm is used to obtain the gradient intensity and gradient direction of each pixel. For each pixel in the image, non-maximum suppression is used to check the gradient intensity of the two neighboring pixels in the gradient direction. If the gradient intensity of the current pixel is greater than that of the two neighboring pixels in the gradient direction, the pixel is retained as an edge point; otherwise, it is suppressed (i.e., the gradient intensity of the pixel is set to 0). After non-maximum suppression, an image containing edge candidate points is obtained, but the intensity of these edge points is different. A double threshold detection method is used to classify edge pixels into three categories: strong edges, weak edges, and non-edges, as follows:

[0112] Set a high threshold and a low threshold; if the gradient strength of a pixel is greater than the high threshold, the pixel is marked as a strong edge point; if the gradient strength of a pixel is between the low threshold and the high threshold, the pixel is marked as a weak edge point; if the gradient strength of a pixel is less than the low threshold, the pixel is considered a non-edge point and is suppressed (i.e., its strength is set to 0).

[0113] After performing dual-threshold detection, weak edge points are further processed through edge tracking, as follows:

[0114] Starting with each strong edge point, check along the surrounding pixels for any weak edge points; if a weak edge point is connected to a strong edge point (i.e., it is adjacent to the strong edge point, or connected to the strong edge point through other weak edge points), then mark the weak edge point as an edge point and retain it; if a weak edge point is not connected to any strong edge point, then consider it noise and suppress it (i.e., set its intensity to 0).

[0115] S 3.2. Based on the generated final edge map, the Harris corner detection method is used to extract corner features from the image. For low-priority static background regions, downsampling is applied to divide the static background region image into blocks for downsampling, resulting in downsampled images. The processing results of high-priority motion regions are combined with the downsampled background regions to form a complete image frame. For motion regions, fine edge and corner information is preserved; for background regions, downsampled brightness and color information are preserved. The corner features of high-priority motion regions and the basic features of low-priority static background regions are combined to form complete feature data.

[0116] S4. Extracting feature points of high-priority motion regions from the optimized image data using optical flow and calculating motion vectors between consecutive frames includes the following steps.

[0117] From the complete feature data, strong corner points in the moving region are extracted as key feature points; based on the optical flow method, the brightness of a key feature point in the image is defined to remain constant across consecutive frames; the optical flow vector is solved within the neighborhood window of the key feature point using the least squares method to obtain the motion vector of each feature point, expressed as:

[0118]

[0119] Where u represents the motion vector component (displacement or velocity) of the feature point in the horizontal axis direction, v represents the motion vector component (displacement or velocity) of the feature point in the vertical axis y direction, m represents the feature point, and W x W represents the brightness gradient (brightness change rate) of an image along the x-axis. In optical flow methods, it is used to describe the change in brightness along the x-axis. y W represents the brightness gradient (brightness change rate) of an image along the y-axis, used in optical flow to describe the change in brightness along the y-axis. t It represents the brightness change of an image over time t (the rate of change in the time dimension), and is used to describe the change in pixel brightness between consecutive frames;

[0120] Based on the motion vector of each feature point, the feature points of the previous frame are matched with the feature points of the current frame to find the position of the same feature point in consecutive frames; by accumulating the optical flow vector of each feature point, the complete motion trajectory of the feature point is generated.

[0121] S5. Based on the calculated motion vectors between consecutive frames, trigger the event-driven frame processing mechanism to automatically adjust the frame rate and processing precision, including the following steps.

[0122] Based on the complete motion trajectory of the feature points, the displacement of the feature points between consecutive frames of the image is determined; based on the displacement between consecutive frames, the speed of the moving object is determined; based on the speed of the moving object, an event-driven frame processing mechanism is triggered to dynamically adjust the frame rate and processing accuracy by setting a speed threshold. When the speed of the moving object is greater than the speed threshold, the frame rate is increased and the image processing accuracy is improved; when the speed of the moving object is less than the speed threshold, the frame rate is reduced and the image processing accuracy is simplified.

[0123] S6. Based on the adjusted frame rate and processing precision, compress the image data and output the compressed image data, including the following steps:

[0124] For high-priority regions that have undergone frame rate adjustment and precision processing, a low compression ratio is used for compression; for low-priority regions that have undergone frame rate adjustment and precision processing, a high compression ratio is used for compression; the compression results of high-priority and low-priority regions are merged to generate compressed complete image data; the motion vector and motion trajectory data of each feature point are encapsulated and merged with the compressed complete image data to obtain a compressed complete image data packet.

[0125] This embodiment also provides an embedded firmware camera data modification device, including: an acquisition and processing module for real-time acquisition of image data and preliminary processing; a segmentation module for dividing the pre-processed image data into regions and assigning priorities using frame difference and background modeling algorithms to generate a region priority mapping table; a layering module for performing layered processing on the image data according to the region priority mapping table to obtain optimized image data and feature information; an extraction module for extracting feature points of high-priority motion regions in the optimized image data using optical flow and calculating motion vectors between consecutive frames; a frame processing module for triggering an event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames to automatically adjust the frame rate and processing precision; and a compression module for compressing the image data based on the adjusted frame rate and processing precision to output compressed image data.

[0126] This embodiment also provides a computer device applicable to the embedded firmware camera data modification method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the embedded firmware camera data modification method proposed in the above embodiment.

[0127] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0128] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the embedded firmware camera data modification method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0129] In summary, this invention achieves precise segmentation of moving and static regions through frame difference and background modeling algorithms, and assigns priority to each region based on its importance, ensuring that the system can prioritize the processing of high-priority moving regions. High-precision edge detection and corner extraction techniques are used for high-priority regions, while downsampling is applied to low-priority regions. This layered processing effectively reduces unnecessary computational overhead and data redundancy. Motion vectors extracted using optical flow methods can calculate the object's speed in real time, and the frame rate and processing accuracy are dynamically adjusted based on the speed, ensuring sufficient detail is captured during rapid motion while saving processing resources during static conditions. Differentiated compression ratios are used for image compression based on region priority, guaranteeing image quality in important regions while reducing storage and transmission overhead in low-priority regions, thus achieving efficient data processing and transmission under limited bandwidth conditions.

[0130] Example 2, referring to Table 1, is the second embodiment of the present invention. To further verify the technical solution of the present invention, experimental simulation data of the embedded firmware camera data modification method are given.

[0131] This experiment employed two approaches: one was the existing embedded camera data processing method, and the other was the improved approach of this invention. The experiment primarily compared image acquisition, processing, compression, and transmission in dynamic scenes.

[0132] In existing technologies, image data captured by cameras typically undergoes a single preprocessing step, including noise reduction and simple histogram equalization, before all image regions are treated with the same processing strategy and compression ratio. This approach fails to differentiate processing for regions of different priorities, leading to severe data redundancy, especially when static background areas occupy a large portion of the frame. Furthermore, existing technologies typically employ a fixed frame rate, failing to adaptively adjust the frame rate and processing precision based on the speed of motion in the scene, further resulting in insufficient detail capture capabilities in fast-moving scenes.

[0133] The improved scheme of this invention introduces the following steps: First, the real-time acquired image data undergoes denoising, histogram equalization, color interpolation, and gamma correction to improve image quality. Then, the image data is divided into moving regions and static background regions using frame difference and background modeling algorithms, and a region priority mapping table is generated based on the importance of the moving regions. For high-priority moving regions, Canny edge detection and Harris corner extraction algorithms are used to ensure sufficient feature information is extracted; low-priority static backgrounds are processed by downsampling to reduce data volume. Next, motion vectors between consecutive frames are calculated using optical flow, and an event-driven frame processing mechanism is triggered based on the speed of moving objects to dynamically adjust the frame rate and processing accuracy. For scenes with fast motion, the system automatically increases the frame rate and processing accuracy; while for static or slow-moving scenes, the frame rate is reduced and processing is simplified to save resources. Finally, the system compresses different image regions using different compression ratios according to priority, ensuring image quality in high-priority regions while effectively reducing data volume in low-priority regions.

[0134] The details are shown in Table 1 below:

[0135] Table 1. Experimental Comparison Table

[0136]

[0137] Firstly, regarding the compression ratio, existing technologies maintain a compression ratio of around 60%-71% for all image regions, lacking a flexible adjustment mechanism. This invention, however, allocates different compression ratios based on the motion characteristics and priorities of the regions. The compression ratio for moving regions is maintained between 86%-91%, ensuring the clarity of moving areas; while for static background regions, the compression ratio is significantly increased to around 94%, reducing unnecessary data transmission. This flexible compression strategy reduces bandwidth consumption while ensuring that critical areas of image quality are preserved.

[0138] Regarding frame rate, the adaptive frame rate adjustment mechanism of this invention demonstrates a significant advantage. Existing technologies typically maintain a fixed frame rate between 9.8 fps and 15.2 fps, failing to handle variations in the speed of moving objects. In this invention, however, the frame rate can dynamically increase to between 25.5 fps and 29.8 fps in moving areas, ensuring detailed capture in fast-moving regions. Simultaneously, in static background areas, the frame rate is reasonably reduced to 14.6 fps, minimizing redundant processing and thus significantly improving system processing efficiency and resource utilization.

[0139] In terms of latency, this invention significantly outperforms existing technologies. Existing technologies have processing latencies between 122.4ms and 144.5ms. This relatively long latency results in poor real-time image processing, especially in motion scenarios, where high latency leads to image quality degradation. This invention, through priority-based layered processing, precise optical flow calculation, and dynamic frame rate adjustment, controls the latency between 81.6ms and 110.1ms, reducing it by approximately 30-40ms. This reduction in latency allows the system to better meet the demands of applications with high real-time requirements, improving the user experience.

[0140] In terms of feature extraction accuracy, this invention also demonstrates significant advantages. Existing technologies have relatively low feature extraction accuracy in static background areas, at only around 37.4%-40%, mainly because they fail to effectively distinguish between moving and static regions, leading to numerous errors in static background areas. In moving regions, the accuracy of existing technologies is between 80.9%-84.5%, indicating room for improvement. This invention employs the Canny edge detection algorithm and the Harris corner extraction algorithm, combined with optical flow to calculate motion vectors between consecutive frames, ensuring accuracy in moving regions and improving the feature extraction accuracy to 82.8%-84.5%. In static background areas, this invention filters out invalid data through downsampling, further improving feature extraction accuracy to 39.7%-40%.

[0141] In summary, this invention significantly improves the processing efficiency and image quality of embedded cameras in dynamic scenes by introducing frame difference method, background modeling algorithm, optical flow method, and adaptive frame rate and compression ratio adjustment strategy.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An embedded firmware camera data modification method, characterized by: The application relates to a method for processing image data, comprising the following steps: Real-time acquisition of image data and preliminary processing thereof; Frame difference method and background modeling algorithm are adopted to divide the image data after preliminary processing into regions and assign priorities to the regions, and a region priority mapping table is generated; the method comprises the following steps: The image data after preliminary processing is stored in a frame buffer to form image frame data; Image data of a current frame and a previous frame in the image frame data is read, and pixel difference between the current frame and the previous frame is calculated for each pixel point; Binary processing is performed on the calculated pixel difference to form a binary image of a preliminary motion region and a static background region; A background modeling algorithm is introduced to distinguish the background state of each pixel point in the binary image in different time periods and construct a background model; The pixel points in the current frame are matched with the background model to determine whether the pixel points in the current frame belong to the background, and the motion region and the static background region are further divided; Priorities are assigned to different regions according to the division result of the motion region and the static background region; Based on the region priority assignment result, a region priority mapping table is generated; According to the region priority mapping table, the image data is processed in layers to obtain optimized image data and feature information; the method comprises the following steps: The generated region priority mapping table and corresponding image frame are acquired, and the image is divided into a high-priority motion region and a low-priority static background region; For the high-priority motion region, a Canny edge detection algorithm is adopted to extract the contour of the moving object and perform edge classification, and a final edge map is generated; Based on the generated final edge map, a Harris corner point detection method is adopted to extract the corner point features in the image; For the low-priority static background region, a downsampling processing mode is adopted to block and downsample the image of the static background region to obtain a downsampled image; The processing result of the high-priority motion region and the downsampled background region of the low-priority static background region are combined to form a complete image frame; The corner point features of the high-priority motion region and the basic features of the low-priority static background region are combined to form complete feature data; Feature points in the high-priority motion region of the optimized image data are extracted by an optical flow method, and motion vectors between continuous frames are calculated; According to the calculated motion vectors between continuous frames, an event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing accuracy; Based on the adjusted frame rate and processing accuracy, the image data is compressed, and the compressed image data is output. The preliminary processing comprises denoising, histogram equalization, color interpolation and gamma correction processing.

2. The method of claim 1, wherein: The feature points in the high-priority motion region of the optimized image data are extracted by the optical flow method, and the motion vectors between continuous frames are calculated, which comprises the following steps:

3. The method of claim 1, wherein: Strong corner points in the motion region are extracted from the complete feature data as key feature points; Based on the optical flow method, the brightness of a certain key feature point in the image is defined to remain unchanged between continuous frames; The optical flow vector is solved in the neighborhood window of the key feature point by the least square method to obtain the motion vector of each feature point, and the expression is: ​ ； wherein u denotes a motion vector component of the feature point in the horizontal coordinate x direction, v denotes a motion vector component of the feature point in the vertical coordinate y direction, m denotes the feature point, denotes a luminance gradient of the image in the horizontal coordinate x direction, denotes a luminance gradient of the image in the vertical coordinate y direction, denotes a luminance change of the image over time t; According to the motion vector of each feature point obtained, the feature points of the previous frame are matched with the feature points of the current frame to find the position of the same feature point in the continuous frames; By accumulating the optical flow vector of each feature point, the complete motion trajectory of the feature point is generated.

4. The method of claim 3, wherein: According to the calculated motion vector between the continuous frames, an event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing accuracy, including the following steps, According to the complete motion trajectory of the feature point, the displacement of the feature point between the continuous frames of the image is determined; Based on the displacement between the continuous frames, the speed of the moving object is judged; Based on the size of the speed of the moving object, an event-driven frame processing mechanism is triggered to dynamically adjust the frame rate and processing accuracy; A speed threshold is set, when the speed of the moving object is greater than the speed threshold, the frame rate is increased and the image processing accuracy is improved; When the speed of the moving object is less than the speed threshold, the frame rate is reduced and the image processing accuracy is simplified.

5. The method of claim 4, wherein: Based on the adjusted frame rate and processing accuracy, the image data is compressed, and the compressed image data is output, including the following steps, For the high-priority area that has undergone frame rate adjustment and accuracy processing, low compression ratio is used for compression; For the low-priority area that has undergone frame rate adjustment and accuracy processing, high compression ratio is used for compression; The compression results of the high-priority area and the low-priority area are merged to generate the complete compressed image data; The motion vector and motion trajectory data of each feature point are packaged and merged with the complete compressed image data to obtain the complete compressed image data packet.

6. An embedded firmware camera data modification apparatus based on any one of claims 1-5, characterized in that: It includes, The acquisition processing module acquires image data in real time and performs preliminary processing; The division module divides and assigns priority to the image data after preliminary processing using frame difference method and background modeling algorithm to generate a region priority mapping table; including the following steps, The image data after preliminary processing is stored in the frame buffer to form image frame data; Read the image data of the current frame and the previous frame in the image frame data, and calculate the pixel difference value between the current frame and the previous frame for each pixel point; The calculated pixel difference value is binarized to form a binary image of the preliminary motion region and the static background region; The background modeling algorithm is introduced to distinguish the background state of each pixel position in the binary image at different time periods to construct a background model; Match the pixel points in the current frame with the background model to determine whether the pixel points in the current frame belong to the background, and further divide the motion region and the static background region; According to the division result of the motion region and the static background region, different regions are assigned priorities; Based on the region priority assignment result, a region priority mapping table is generated; The hierarchical module processes the image data according to the region priority mapping table to obtain optimized image data and feature information; including the following steps, Obtain the generated region priority mapping table and the corresponding image frame, and divide the image into high-priority motion regions and low-priority static background regions; For the high-priority motion region, the Canny edge detection algorithm is used to extract the contour of the moving object and perform edge classification to generate the final edge map; Based on the generated final edge map, a Harris corner point detection method is used to extract the corner point features in the image; For the low-priority static background area, a downsampling processing method is used to block and downsample the image of the static background area to obtain a downsampled image; The high-priority motion area processing result and the low-priority downsampled background area are combined to form a complete image frame; The corner point features of the high-priority motion area and the basic features of the low-priority static background area are combined to form complete feature data; The extraction module extracts the feature points of the high-priority motion area in the optimized image data by using an optical flow method, and calculates the motion vectors between consecutive frames; The frame processing module triggers an event-driven frame processing mechanism according to the calculated motion vectors between consecutive frames, and automatically adjusts the frame rate and processing accuracy; The compression module compresses the image data based on the adjusted frame rate and processing accuracy, and outputs the compressed image data. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the embedded firmware camera data modification method of any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the embedded firmware camera data modification method of any one of claims 1-5.

Images