Embedded firmware camera data modification method and device, medium and equipment

Abstract

The invention discloses an embedded firmware camera data modification method and device, a medium and equipment, and relates to the technical field of computer vision, and the method comprises the steps: collecting image data in real time, and carrying out the primary processing; a frame difference method and a background modeling algorithm are adopted to carry out region division and priority distribution on the image data after primary processing, and a region priority mapping table is generated; extracting feature points of a high-priority motion area in the optimized image data through an optical flow method, and calculating a motion vector between continuous frames; according to the calculated motion vector between the continuous frames, an event-driven frame processing mechanism is triggered, and the frame rate and the processing precision are automatically adjusted; through the motion vector extracted through the optical flow method, the motion speed of the object can be calculated in real time, the frame rate and the processing precision are dynamically adjusted according to the motion speed, it is ensured that enough details are captured in the rapid motion process, and processing resources are saved in the static process.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an embedded firmware camera data modification method, device, medium and equipment. Background Art

[0002] With the rapid development of computer vision and image processing technology, the demand for camera data processing in embedded systems is becoming increasingly complex. Traditional embedded cameras are mainly used for image acquisition and transmission of static scenes, and rarely involve the processing of real-time dynamic scenes. However, with the expansion of fields such as the Internet of Things, intelligent monitoring and autonomous driving, the demand for data processing in real-time dynamic scenes is becoming more urgent. Existing embedded camera systems lack flexible frame rate adjustment and regional priority processing mechanisms, and are usually unable to efficiently process large amounts of dynamic scene data under limited bandwidth. At the same time, existing technologies mostly use image compression methods with fixed compression ratios and processing accuracy, which cannot be flexibly adjusted according to scene changes, resulting in low data transmission efficiency and loss of important information.

[0003] The shortcomings of the existing technology are mainly manifested in two aspects: first, the existing embedded camera system lacks a differentiated processing mechanism for different areas when processing dynamic scenes, and adopts a unified processing strategy for the entire image data, which cannot effectively distinguish between important motion areas and static background areas, resulting in a waste of resources; second, the existing system cannot dynamically adjust the frame rate and processing accuracy according to the speed of the moving object, resulting in the inability to capture enough details in fast-moving scenes, and too much redundant data in static scenes. Therefore, there is an urgent need for an embedded camera system that can dynamically adjust the frame rate, process image data in layers, and perform efficient compression. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an embedded firmware camera data modification method to solve the problems in existing embedded camera systems of lack of differentiated processing of dynamic scenes and inability to adaptively adjust frame rate and processing accuracy.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an embedded firmware camera data modification method, which comprises:

[0008] Collect image data in real time and perform preliminary processing;

[0009] The frame difference method and background modeling algorithm are used to divide the preliminarily processed image data into regions and assign priorities to generate a regional priority mapping table;

[0010] According to the regional priority mapping table, the image data is processed in layers to obtain optimized image data and feature information;

[0011] The optical flow method is used to extract feature points of high-priority motion areas in the optimized image data and calculate motion vectors between consecutive frames;

[0012] According to the calculated motion vectors between consecutive frames, the event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing accuracy;

[0013] Based on the adjusted frame rate and processing accuracy, the image data is compressed, and the compressed image data is output.

[0014] As a preferred solution of the embedded firmware camera data modification method of the present invention, the preliminary processing includes denoising, histogram equalization, color interpolation and gamma correction processing.

[0015] As a preferred solution of the embedded firmware camera data modification method of the present invention, the frame difference method and background modeling algorithm are used to divide the image data after preliminary processing into regions and assign priorities, and the generation of the regional priority mapping table includes the following steps:

[0016] The image data after preliminary processing is stored in a frame buffer to form image frame data;

[0017] Read the image data of the current frame and the previous frame in the image frame data, and calculate the pixel difference between the current frame and the previous frame for each pixel point;

[0018] The calculated pixel difference is binarized to form a preliminary binary image of the motion area and the static background area;

[0019] A background modeling algorithm is introduced to distinguish the background states in different time periods for each pixel position in the binary image and construct a background model.

[0020] Match the pixels in the current frame with the background model to determine whether the pixels in the current frame belong to the background, and further divide the moving area and the static background area;

[0021] According to the results of the motion area and static background area division, priorities are assigned to different areas;

[0022] Based on the regional priority allocation result, a regional priority mapping table is generated.

[0023] As a preferred solution of the embedded firmware camera data modification method of the present invention, wherein: according to the regional priority mapping table, the image data is layered processed to obtain the optimized image data and feature information, including the following steps:

[0024] Obtain the generated region priority mapping table and the corresponding image frame, and divide the image into a high-priority motion region and a low-priority static background region;

[0025] For high-priority motion areas, the Canny edge detection algorithm is used to extract the contours of moving objects and perform edge classification to generate the final edge map;

[0026] Based on the generated final edge map, the Harris corner detection method is used to extract the corner features in the image;

[0027] For the low-priority static background area, a downsampling processing method is adopted to downsample the image of the static background area in blocks to obtain a downsampled image;

[0028] The processing result of the high-priority motion region is combined with the downsampled background region of low priority to form a complete image frame;

[0029] The corner point features of the high-priority motion area and the basic features of the low-priority static background area are combined to form complete feature data.

[0030] As a preferred solution of the embedded firmware camera data modification method of the present invention, wherein: extracting feature points of high-priority motion areas in the optimized image data by optical flow method, and calculating motion vectors between consecutive frames include the following steps:

[0031] Extract stronger corner points in the motion area from the complete feature data as key feature points;

[0032] Based on the optical flow method, the brightness of a key feature point in the image is defined to remain unchanged between consecutive frames;

[0033] The optical flow vector is solved in the neighborhood window of the key feature point by the least squares method to obtain the motion vector of each feature point. The expression is:

[0034]

[0035] Among them, u represents the motion vector component of the feature point in the horizontal coordinate x direction, v represents the motion vector component of the feature point in the vertical coordinate y direction, m represents the feature point, I x Represents the brightness gradient of the image in the x-direction, I y Represents the brightness gradient of the image in the y direction of the vertical coordinate, I t Represents the brightness change of the image at time t;

[0036] According to the motion vector of each feature point obtained, the feature point of the previous frame is matched with the feature point of the current frame, and the position of the same feature point is found in the consecutive frames;

[0037] By accumulating the optical flow vector of each feature point, the complete motion trajectory of the feature point is generated.

[0038] As a preferred solution of the embedded firmware camera data modification method of the present invention, wherein: according to the calculated motion vector between consecutive frames, triggering the event-driven frame processing mechanism, automatically adjusting the frame rate and processing accuracy includes the following steps:

[0039] According to the complete motion trajectory of the feature point, the displacement of the feature point between consecutive frames of the image is determined;

[0040] Determine the speed of a moving object based on the displacement between consecutive frames;

[0041] Based on the speed of the moving object, the event-driven frame processing mechanism is triggered to dynamically adjust the frame rate and processing accuracy;

[0042] Set a speed threshold. When the speed of the moving object is greater than the speed threshold, increase the frame rate and improve the image processing accuracy.

[0043] When the speed of the moving object is less than the speed threshold, the frame rate is reduced and the image processing accuracy is simplified.

[0044] As a preferred solution of the embedded firmware camera data modification method of the present invention, the image data is compressed based on the adjusted frame rate and processing accuracy, and the compressed image data is outputted, which includes the following steps:

[0045] For high-priority areas that have undergone frame rate adjustment and precision processing, a low compression ratio is used for compression;

[0046] For low-priority areas that have undergone frame rate adjustment and precision processing, high compression ratio is used for compression;

[0047] Merge the compression results of the high priority area and the low priority area to generate compressed complete image data;

[0048] The motion vector and motion trajectory data of each feature point are encapsulated and merged with the compressed complete image data to obtain a compressed complete image data packet.

[0049] In a second aspect, the present invention provides an embedded firmware camera data modification device, comprising:

[0050] The acquisition and processing module acquires image data in real time and performs preliminary processing;

[0051] The division module uses the frame difference method and background modeling algorithm to divide the preliminarily processed image data into regions and assign priorities to generate a regional priority mapping table;

[0052] The stratification module processes the image data in layers according to the regional priority mapping table to obtain optimized image data and feature information;

[0053] An extraction module, which extracts feature points of high-priority motion areas in the optimized image data by optical flow method and calculates motion vectors between consecutive frames;

[0054] The frame processing module triggers the event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames, and automatically adjusts the frame rate and processing accuracy;

[0055] The compression module compresses the image data based on the adjusted frame rate and processing accuracy, and outputs the compressed image data.

[0056] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the embedded firmware camera data modification method as described in the first aspect of the present invention is implemented.

[0057] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the embedded firmware camera data modification method as described in the first aspect of the present invention is implemented.

[0058] The beneficial effects of the present invention are as follows: accurate division of moving areas and static areas is achieved through the frame difference method and background modeling algorithm, and a priority is assigned to each area according to its importance, ensuring that the system can give priority to high-priority moving areas; high-precision edge detection and corner point extraction technology is used for high-priority areas, while downsampling is performed on low-priority areas, and layered processing effectively reduces unnecessary computing overhead and data redundancy; the motion vector extracted by the optical flow method can calculate the movement speed of the object in real time, and dynamically adjust the frame rate and processing accuracy according to the movement speed, ensuring that sufficient details are captured during rapid movement, while saving processing resources when stationary; differentiated compression ratios are used for image compression according to regional priorities, which not only ensures the image quality of important areas, but also reduces the storage and transmission overhead of low-priority areas, thereby achieving efficient data processing and transmission under limited bandwidth conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0060] Figure 1 This is a flow chart of the embedded firmware camera data modification method in Example 1.

[0061] Figure 2 This is a schematic diagram of the embedded firmware camera data modification system in Example 1. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0063] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0064] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0065] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides an embedded firmware camera data modification method, comprising the following steps:

[0066] S1. Collect image data in real time and perform preliminary processing.

[0067] S1.1. Preliminary processing includes denoising, histogram equalization, color interpolation and gamma correction.

[0068] Specifically, denoising refers to smoothing out variations in brightness or color in an image, removing small random fluctuations.

[0069] After the denoising process is completed, histogram equalization is performed on each frame of the image to enhance the contrast of the image. The operation steps are as follows:

[0070] For grayscale images, the frequency of occurrence of each grayscale level is counted to obtain a grayscale histogram, which is expressed as:

[0071]

[0072] Where J(i) represents the frequency of grayscale value i in the image, x represents the horizontal coordinate of the pixel, y represents the vertical coordinate of the pixel, M represents the width of the image, N represents the height of the image, i represents the grayscale value, δ(I gray (x,y)-i) represents the Kronecker function, I gray (x,y) represents the grayscale value at the pixel coordinate (x,y);

[0073] Furthermore, the expression of the Kronecker function is:

[0074]

[0075] If I gray (x, y) = I, the function value is 1, indicating that the gray value at the pixel coordinate (X, y) belongs to gray value i. gray (x,y)≠i, the function value is 0, indicating that the gray value at the pixel coordinate (x,y) does not belong to the gray value i;

[0076] Through the cumulative distribution function, the probability of each gray value occurring is accumulated to obtain the cumulative probability, which is expressed as:

[0077]

[0078] Among them, CDF(i) represents the cumulative probability of gray value i, j represents the gray value index, and H(j) represents the frequency of occurrence of the jth gray value in the image;

[0079] According to the cumulative probability, the gray value of each pixel in the original image is remapped to a new gray value. The expression is:

[0080]

[0081] Among them, I' gray (x, y) represents the gray value after remapping at the pixel coordinate (x, y), and CDF(I(x, y)) represents the gray value I at the pixel coordinate (x, y) gray Cumulative probability of (x,y), CDF min represents the minimum cumulative probability, E represents the total number of pixels in the image, and L represents the number of gray levels.

[0082] Color interpolation refers to the use of the classic trilinear interpolation algorithm to convert monochrome pixels into a complete three-channel color image. The expression is:

[0083]

[0084] Among them, G represents green, R represents red, B represents blue, and H represents RGB(x, y) represents the RGB three-channel value at the pixel coordinate (x, y), H RGB (x-1, Y) represents the RGB three-channel value at the adjacent pixel coordinate (x-1, y), H RGB (x+1, y) represents the RGB three-channel value at the adjacent pixel coordinate (x+1, Y), H RGB (x, Y-1) represents the RGB three-channel value at the adjacent pixel coordinate (X, Y-1), H RGB (x, y+1) represents the RGB three-channel value at the adjacent pixel coordinate (x, Y+1);

[0085] After color interpolation, the brightness of the image is still linear and is not suitable for direct display or further processing. In order to make the image brightness more consistent with the nonlinear response of the human visual system, the image needs to be gamma corrected. The expression is:

[0086]

[0087] Among them, L cor (x, y) represents the color value after gamma correction at the pixel coordinates (x, y). γ represents the parameter of gamma correction, which indicates the nonlinearity of the human eye's perception of brightness. The common γ value is 2.2, which means that the human eye is more sensitive to darker areas than to brighter areas.

[0088] S2. Use the frame difference method and background modeling algorithm to divide the preliminarily processed image data into regions and assign priorities to generate a regional priority mapping table.

[0089] S2.1. The image data after preliminary processing is stored in a frame buffer to form image frame data; the image data of the current frame and the previous frame in the image frame data are read, and the pixel difference between the current frame and the previous frame is calculated for each pixel point.

[0090] Specifically, the expression of pixel difference is:

[0091] D(x,y)=|Q t (x,y)-Q t-1 (x,y)|;

[0092] Where D(x,y) represents the pixel difference at the pixel coordinate (x,y). A larger difference indicates that the pixel may belong to the motion area, while a smaller difference indicates that the pixel may belong to the static background area. t (x, y) represents the pixel value at the pixel coordinate (x, y) at time t. The pixel value is the grayscale value or the RGB three-channel value. Q t-1 (x,y) represents the pixel value at the pixel coordinate (x,y) at time t-1.

[0093] S2.2. Binarize the calculated pixel differences to form preliminary binary images of the motion area and the static background area.

[0094] Specifically, the expression of binarization processing is:

[0095]

[0096] Among them, P(x,y) represents the binary result at the pixel coordinates (x,y), 1 indicates that the pixel belongs to the moving area, 0 indicates that the pixel belongs to the static background area, T represents the threshold for distinguishing the moving area from the static background area, and when the pixel difference D(x,y) exceeds the threshold T, the pixel is considered to be moving; otherwise, the pixel is considered to be stationary.

[0097] S2.3. Introduce a background modeling algorithm to distinguish the background states in different time periods for each pixel position in the binary image and build a background model; match the pixel points in the current frame with the background model to determine whether the pixel points in the current frame belong to the background, and further divide the moving area and the static background area.

[0098] Specifically, the background model is usually composed of multiple Gaussian distributions. A common practice is to use a Gaussian mixture model (GMM) to model the background of each pixel. If the pixel value in the current frame matches a Gaussian distribution in the background model (that is, the pixel value is close enough to the mean of the Gaussian distribution), the mean and variance of the Gaussian distribution need to be updated.

[0099] Furthermore, the mean update expression of the background model is:

[0100]

[0101] in, Indicates the updated k The mean of a Gaussian distribution, α represents the update rate of the background model, Indicates the number before the update k The mean of a Gaussian distribution, k Represents the Gaussian distribution index;

[0102] The variance update expression of the background model is:

[0103]

[0104] in, Indicates the updated k The variance of a Gaussian distribution, Indicates the number before the update kThe variance of a Gaussian distribution;

[0105] According to the update result of the background model, it is determined whether the current pixel belongs to the background. If a pixel does not conform to any Gaussian distribution, it is considered that the pixel belongs to the foreground area (i.e., the motion area); otherwise, the pixel is considered to belong to the background area.

[0106] S2.4. According to the results of the division of the motion area and the static background area, priorities are assigned to different areas; based on the results of the area priority assignment, a regional priority mapping table is generated.

[0107] Specifically, the priority allocation follows the following rules:

[0108] For the detected motion area, a higher priority is assigned according to the relative importance; for the static background area, a lower priority is assigned.

[0109] S 3. According to the regional priority mapping table, the image data is processed in layers to obtain optimized image data and feature information.

[0110] S 3.1. Obtain the generated regional priority mapping table and the corresponding image frame, and divide the image into high-priority motion areas and low-priority static background areas; for the high-priority motion areas, use the Canny edge detection algorithm to extract the contours of the moving objects and perform edge classification to generate the final edge map.

[0111] Specifically, the Canny edge detection algorithm is used to obtain the gradient strength and gradient direction of each pixel; for each pixel in the image, the non-maximum suppression method is used to check the gradient strength of the two neighboring pixels of the pixel in the gradient direction according to its gradient direction; if the gradient strength of the current pixel is greater than the two neighboring pixels in its gradient direction, the pixel is retained as an edge point; otherwise, it is suppressed (that is, the gradient strength of the pixel is set to 0); after non-maximum suppression, an image containing edge candidate points is obtained, but the strengths of these edge points are different. The double threshold detection method is used to divide the edge pixels into three categories: strong edge, weak edge and non-edge. The operation is as follows:

[0112] Set a high threshold and a low threshold; if the gradient strength of a pixel is greater than the high threshold, the pixel is marked as a strong edge point; if the gradient strength of a pixel is between the low threshold and the high threshold, the pixel is marked as a weak edge point; if the gradient strength of a pixel is less than the low threshold, the pixel is considered a non-edge point and is suppressed (i.e., its strength is set to 0).

[0113] After performing dual threshold detection, the weak edge points are further processed by edge tracking, as follows:

[0114] Starting from each strong edge point, check along the pixels around it whether there are any weak edge points; if a weak edge point is connected to a strong edge point (that is, it is adjacent to the strong edge point, or connected to the strong edge point through other weak edge points), then the weak edge point is also marked as an edge point and retained; if a weak edge point is not connected to any strong edge point, it is considered to be noise and is suppressed (that is, its intensity is set to 0).

[0115] S 3.2. Based on the generated final edge map, the Harris corner detection method is used to extract the corner features in the image; for the low-priority static background area, the downsampling processing method is used to downsample the image of the static background area in blocks to obtain the downsampled image; the processing result of the high-priority motion area is combined with the low-priority downsampled background area to form a complete image frame; for the motion area, the fine edge and corner information is retained; for the background area, the brightness and color information after downsampling is retained. The corner features of the high-priority motion area and the basic features of the low-priority static background area are combined to form complete feature data.

[0116] S 4. Extracting feature points of high-priority motion areas in the optimized image data by optical flow method, and calculating motion vectors between consecutive frames includes the following steps:

[0117] The stronger corner points in the motion area are extracted from the complete feature data as key feature points; based on the optical flow method, the brightness of a key feature point in the image is defined to remain unchanged between consecutive frames; the optical flow vector is solved in the neighborhood window of the key feature point by the least squares method to obtain the motion vector of each feature point, which is expressed as:

[0118]

[0119] Where u represents the motion vector component (displacement or velocity) of the feature point in the horizontal direction, v represents the motion vector component (displacement or velocity) of the feature point in the vertical direction, m represents the feature point, and W x Indicates the brightness gradient (brightness change rate) of the image in the x-direction of the horizontal coordinate. It is used to describe the change of brightness in the x-direction of the horizontal coordinate in the optical flow method. y Indicates the brightness gradient (brightness change rate) of the image in the y-direction of the vertical coordinate. It is used to describe the change of brightness in the y-direction of the vertical coordinate in the optical flow method. t Represents the brightness change of an image at time t (the rate of change in the time dimension), which is used to describe the change in pixel brightness between consecutive frames;

[0120] According to the motion vector of each feature point obtained, the feature point of the previous frame is matched with the feature point of the current frame, and the position of the same feature point is found in consecutive frames; by accumulating the optical flow vector of each feature point, the complete motion trajectory of the feature point is generated.

[0121] S 5. According to the calculated motion vectors between consecutive frames, triggering the event-driven frame processing mechanism, automatically adjusting the frame rate and processing accuracy includes the following steps:

[0122] According to the complete motion trajectory of the feature points, the displacement of the feature points between consecutive frames of the image is determined; based on the displacement between consecutive frames, the speed of the moving object is judged; based on the speed of the moving object, the event-driven frame processing mechanism is triggered, and the frame rate and processing accuracy are dynamically adjusted to set the speed threshold. When the speed of the moving object is greater than the speed threshold, the frame rate is increased and the image processing accuracy is improved; when the speed of the moving object is less than the speed threshold, the frame rate is reduced and the image processing accuracy is simplified.

[0123] S6. Based on the adjusted frame rate and processing accuracy, compress the image data, and output the compressed image data, including the following steps:

[0124] For high-priority areas that have undergone frame rate adjustment and precision processing, a low compression ratio is used for compression; for low-priority areas that have undergone frame rate adjustment and precision processing, a high compression ratio is used for compression; the compression results of the high-priority areas and the low-priority areas are merged to generate compressed complete image data; the motion vector and motion trajectory data of each feature point are encapsulated and merged with the compressed complete image data to obtain a compressed complete image data packet.

[0125] The present embodiment also provides an embedded firmware camera data modification device, including: an acquisition and processing module, which acquires image data in real time and performs preliminary processing; a division module, which uses a frame difference method and a background modeling algorithm to divide the image data after preliminary processing into regions and assign priorities, and generate a regional priority mapping table; a stratification module, which performs stratified processing on the image data according to the regional priority mapping table to obtain optimized image data and feature information; an extraction module, which extracts feature points of high-priority motion areas in the optimized image data through an optical flow method, and calculates motion vectors between consecutive frames; a frame processing module, which triggers an event-driven frame processing mechanism according to the calculated motion vectors between consecutive frames, and automatically adjusts the frame rate and processing accuracy; a compression module, which compresses the image data based on the adjusted frame rate and processing accuracy, and outputs the compressed image data.

[0126] This embodiment also provides a computer device, which is suitable for the case of an embedded firmware camera data modification method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the embedded firmware camera data modification method proposed in the above embodiment.

[0127] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0128] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for modifying embedded firmware camera data proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0129] In summary, the present invention realizes the accurate division of moving areas and static areas through the frame difference method and background modeling algorithm, and assigns priority to each area according to its importance, ensuring that the system can give priority to high-priority moving areas; high-precision edge detection and corner extraction technology are used for high-priority areas, while downsampling is performed on low-priority areas, and the layered processing effectively reduces unnecessary computing overhead and data redundancy; the motion vector extracted by the optical flow method can calculate the movement speed of the object in real time, and dynamically adjust the frame rate and processing accuracy according to the movement speed, ensuring that sufficient details are captured during fast movement and saving processing resources when stationary; differentiated compression ratios are used for image compression according to regional priorities, which not only ensures the image quality of important areas, but also reduces the storage and transmission overhead of low-priority areas, thereby achieving efficient data processing and transmission under limited bandwidth conditions.

[0130] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of the embedded firmware camera data modification method are provided.

[0131] This experiment adopts two solutions: one is the embedded camera data processing method of the prior art, and the other is the improved solution of the present invention. The experiment mainly compares image acquisition, processing, compression and transmission in dynamic scenes.

[0132] In the prior art, the image data collected by the camera usually goes through a single preprocessing process, including denoising and simple histogram equalization, and then the same processing strategy and compression ratio are applied to all image regions. This solution cannot perform differentiated processing for regions of different priorities, resulting in serious data redundancy problems, especially when the static background area occupies most of the screen. In addition, the prior art usually uses a fixed frame rate and cannot adaptively adjust the frame rate and processing accuracy according to the speed of movement in the scene, further resulting in insufficient ability to capture details in fast-moving scenes.

[0133] The improved scheme of the present invention introduces the following steps: First, the real-time collected image data is processed by denoising, histogram equalization, color interpolation and gamma correction to improve the picture quality. Subsequently, the frame difference method and background modeling algorithm are used to divide the image data into motion areas and static background areas, and the priorities are assigned according to the importance of the motion areas to generate a regional priority mapping table. For high-priority motion areas, Canny edge detection and Harris corner point extraction algorithms are used to ensure that sufficient feature information is extracted; the low-priority static background is processed by downsampling to reduce the amount of data. Then, the motion vector between consecutive frames is calculated by the optical flow method, and the event-driven frame processing mechanism is triggered based on the speed of the moving object to dynamically adjust the frame rate and processing accuracy. For scenes with faster motion speed, the system automatically increases the frame rate and processing accuracy; for scenes with static or slow motion, the frame rate is reduced and the processing is simplified to save resources. Finally, the system compresses different image areas with different compression ratios according to the priority, ensuring the image quality of the high-priority area and effectively reducing the amount of data in the low-priority area.

[0134] The details are shown in Table 1 below:

[0135] Table 1 Experimental comparison table

[0136]

[0137] First, in terms of compression ratio, the existing technology maintains the compression ratio of all image regions at about 60%-71%, lacking a flexible adjustment mechanism. The present invention allocates different compression ratios according to the motion characteristics and priorities of the regions, and the compression ratio of the motion region is maintained between 86%-91%, ensuring the clarity of the motion region; while for the static background region, the compression ratio is significantly increased to about 94%, reducing unnecessary transmission data. This flexible compression strategy reduces bandwidth consumption while ensuring that the key areas of image quality are preserved.

[0138] In terms of frame rate, the adaptive frame rate adjustment mechanism of the present invention shows obvious advantages. The frame rate of the prior art is basically fixed between 9.8fps and 15.2fps, which cannot cope with the speed changes of moving objects. In the present invention, the frame rate can be dynamically increased to 25.5fps to 29.8fps in the motion area, ensuring the capture of details in the fast-moving area. At the same time, in the static background area, the frame rate is reasonably reduced to 14.6fps, reducing redundant processing, thereby greatly improving the processing efficiency and resource utilization of the system.

[0139] In terms of latency, the performance of the present invention is significantly better than that of the prior art. The processing latency of the prior art is between 122.4ms and 144.5ms. The longer latency makes the real-time performance of image processing poor, especially in motion scenes, where higher latency will lead to reduced image quality. The present invention controls the latency between 81.6ms and 110.1ms through priority layered processing, precise calculation of the optical flow method, and dynamic frame rate adjustment, reducing it by about 30-40ms. This reduction in latency enables the system to better meet application scenarios with higher real-time requirements and improves the user experience.

[0140] In terms of feature extraction accuracy, the present invention also shows great advantages. The feature extraction accuracy of the prior art is low in the static background area, only about 37.4%-40%, mainly because the prior art fails to effectively distinguish between the moving area and the static area, resulting in more errors in the static background area. In the moving area, the accuracy of the prior art is between 80.9%-84.5%, and there is still room for improvement. The present invention adopts the Canny edge detection algorithm and the Harris corner point extraction algorithm, combined with the optical flow method to calculate the motion vector between consecutive frames, thereby ensuring the accuracy of the moving area and improving the feature extraction accuracy to 82.8%-84.5%. In the static background area, the present invention filters out invalid data through downsampling processing, further improving the accuracy of feature extraction to 39.7%-40%.

[0141] In summary, the present invention significantly improves the processing efficiency and image quality of embedded cameras in dynamic scenes by introducing frame difference method, background modeling algorithm, optical flow method, and adaptive frame rate and compression ratio adjustment strategy.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for modifying data of an embedded firmware camera, characterized in that: include, Collect image data in real time and perform preliminary processing; The frame difference method and background modeling algorithm are used to divide the preliminarily processed image data into regions and assign priorities to generate a regional priority mapping table; According to the regional priority mapping table, the image data is processed in layers to obtain optimized image data and feature information; The optical flow method is used to extract feature points of high-priority motion areas in the optimized image data and calculate motion vectors between consecutive frames; According to the calculated motion vectors between consecutive frames, the event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing accuracy; Based on the adjusted frame rate and processing accuracy, the image data is compressed, and the compressed image data is output.

2. The embedded firmware camera data modification method as claimed in claim 1, characterized in that: The preliminary processing includes denoising, histogram equalization, color interpolation and gamma correction processing.

3. The embedded firmware camera data modification method as claimed in claim 2, characterized in that: The frame difference method and background modeling algorithm are used to divide the preliminarily processed image data into regions and assign priorities. The generation of the regional priority mapping table includes the following steps: The image data after preliminary processing is stored in a frame buffer to form image frame data; Read the image data of the current frame and the previous frame in the image frame data, and calculate the pixel difference between the current frame and the previous frame for each pixel point; The calculated pixel difference is binarized to form a preliminary binary image of the motion area and the static background area; A background modeling algorithm is introduced to distinguish the background states in different time periods for each pixel position in the binary image and construct a background model. Match the pixels in the current frame with the background model to determine whether the pixels in the current frame belong to the background, and further divide the moving area and the static background area; According to the results of the motion area and static background area division, priorities are assigned to different areas; Based on the regional priority allocation result, a regional priority mapping table is generated.

4. The embedded firmware camera data modification method as claimed in claim 3, characterized in that: According to the regional priority mapping table, the image data is processed in layers to obtain optimized image data and feature information. The following steps are included: Obtain the generated region priority mapping table and the corresponding image frame, and divide the image into a high-priority motion region and a low-priority static background region; For high-priority motion areas, the Canny edge detection algorithm is used to extract the contours of moving objects and perform edge classification to generate the final edge map; Based on the generated final edge map, the Harris corner detection method is used to extract the corner features in the image; For the low-priority static background area, a downsampling processing method is adopted to downsample the image of the static background area in blocks to obtain a downsampled image; The processing result of the high-priority motion region is combined with the downsampled background region of low priority to form a complete image frame; The corner point features of the high-priority motion area and the basic features of the low-priority static background area are combined to form complete feature data.

5. The embedded firmware camera data modification method as claimed in claim 4, characterized in that: Extracting feature points of high-priority motion areas in optimized image data by optical flow method and calculating motion vectors between consecutive frames include the following steps: Extract stronger corner points in the motion area from the complete feature data as key feature points; Based on the optical flow method, the brightness of a key feature point in the image is defined to remain unchanged between consecutive frames; The optical flow vector is solved in the neighborhood window of the key feature point by the least squares method to obtain the motion vector of each feature point. The expression is: Among them, u represents the motion vector component of the feature point in the horizontal coordinate x direction, v represents the motion vector component of the feature point in the vertical coordinate y direction, m represents the feature point, I x Represents the brightness gradient of the image in the x-direction, I y Represents the brightness gradient of the image in the y direction of the vertical coordinate, I t Represents the brightness change of the image at time t; According to the motion vector of each feature point obtained, the feature point of the previous frame is matched with the feature point of the current frame, and the position of the same feature point is found in the consecutive frames; By accumulating the optical flow vector of each feature point, the complete motion trajectory of the feature point is generated.

6. The embedded firmware camera data modification method as claimed in claim 5, characterized in that: According to the calculated motion vectors between consecutive frames, the event-driven frame processing mechanism is triggered to automatically adjust the frame rate and processing accuracy, including the following steps: According to the complete motion trajectory of the feature point, the displacement of the feature point between consecutive frames of the image is determined; Determine the speed of a moving object based on the displacement between consecutive frames; Based on the speed of the moving object, the event-driven frame processing mechanism is triggered to dynamically adjust the frame rate and processing accuracy; Set a speed threshold. When the speed of the moving object is greater than the speed threshold, increase the frame rate and improve the image processing accuracy. When the speed of the moving object is less than the speed threshold, the frame rate is reduced and the image processing accuracy is simplified.

7. The embedded firmware camera data modification method as claimed in claim 6, characterized in that: Based on the adjusted frame rate and processing accuracy, compressing the image data, and outputting the compressed image data includes the following steps: For high-priority areas that have undergone frame rate adjustment and precision processing, a low compression ratio is used for compression; For low-priority areas that have undergone frame rate adjustment and precision processing, high compression ratio is used for compression; Merge the compression results of the high priority area and the low priority area to generate compressed complete image data; The motion vector and motion trajectory data of each feature point are encapsulated and merged with the compressed complete image data to obtain a compressed complete image data packet.

8. An embedded firmware camera data modification device, based on the embedded firmware camera data modification method according to any one of claims 1 to 7, characterized in that: include, The acquisition and processing module acquires image data in real time and performs preliminary processing; The division module uses the frame difference method and background modeling algorithm to divide the preliminarily processed image data into regions and assign priorities to generate a regional priority mapping table; The stratification module processes the image data in layers according to the regional priority mapping table to obtain optimized image data and feature information; An extraction module, which extracts feature points of high-priority motion areas in the optimized image data by optical flow method and calculates motion vectors between consecutive frames; The frame processing module triggers the event-driven frame processing mechanism based on the calculated motion vectors between consecutive frames, and automatically adjusts the frame rate and processing accuracy; The compression module compresses the image data based on the adjusted frame rate and processing accuracy, and outputs the compressed image data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the embedded firmware camera data modification method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the embedded firmware camera data modification method according to any one of claims 1 to 7 are implemented.

Images