Image recognition system and image recognition sensor applied to camera

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image recognition system and an image recognition sensor applied to a camera, and the system comprises an image recognition module, a preprocessing module, a texture recognition module, an optical flow track module and a camera tracking module which are in communication connection in sequence. FPGA hardware acceleration is combined with redundancy reduction of particle screening, so that an image recognition system can stably perform full-link processing on a high-speed camera, and the non-delay tracking requirement of a high-speed moving target is met; through a texture channel fused by multiple edge channels, the edge integrity rate in a high-speed motion fuzzy scene is improved, and the tracking loss rate of light field particles is reduced; through a designed double-constraint optical flow mechanism, the optical flow estimation error of a high-speed moving target is reduced, the inter-frame position deviation of camera tracking is reduced, and accurate positioning of the high-speed target is realized, so that the problems of image precision reduction and tracking breakage caused by high-speed movement of the target in high-speed camera image recognition are solved.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to an image recognition system and image recognition sensor for use in cameras. Background Technology

[0002] The importance of high-speed camera image recognition systems in modern technological applications is self-evident. These systems are widely used in numerous fields, such as industrial automation, intelligent transportation systems, aerospace, and military reconnaissance. Accurately and rapidly identifying high-speed moving targets is a key technology for achieving optimal high-speed camera image quality.

[0003] However, existing high-speed camera image processing systems face numerous challenges in recognizing high-speed moving targets. Due to the extremely high speed of the target, traditional processing methods often struggle to process and recognize images in real time. This results in the lack of compensation for the high-speed motion characteristics of the target when processing continuous high-speed images. The inter-frame positional deviation of the target being tracked by the camera due to high-speed movement causes local blurring in the image, thus affecting the accuracy of image recognition. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides an image recognition system and image recognition sensor for use in cameras. This system solves the problem of inter-frame positional deviations in camera tracking caused by the high-speed movement of the target when the image processing system of a high-speed camera is dealing with the recognition of high-speed moving targets, which in turn causes local blurring of the image.

[0005] The objective of this invention can be achieved through the following technical solution: an image recognition system applied to a camera, comprising: The image recognition module is used to acquire image information and extract at least one target object from the acquired image information. The image algorithm based on FPGA performs pixel scanning on the image information to obtain a grayscale image. The image information includes continuous frame images. The preprocessing module uses a corner detection algorithm based on grayscale images to obtain the corner points of each target object, and filters the corner points of each target image to obtain the image light field particles of each target image. The position changes of each image light field particle in consecutive frames are tracked to obtain image light field particle motion trend data; The texture recognition module is used to perform edge visual feature analysis on each target image, generate multiple edge visual channels, and generate texture channels for each target image based on the edge visual channels. The optical flow trajectory module is used to perform optical flow displacement trajectory analysis on the motion trend data of light field particles based on the image texture channel, so as to obtain the optical flow displacement trajectory of each target image; The camera tracking module is used to perform subsequent positioning and tracking of each image light field particle based on the optical flow displacement trajectory using an image tracking algorithm.

[0006] Preferably, the texture recognition module includes an edge recognition unit and a texture recognition unit; The edge recognition unit uses the multi-scale Canny edge detection algorithm to perform edge visual feature analysis: by setting high and low threshold gradients, the edge contours of each target object are extracted in multiple scale spaces, and the multi-scale edge information is fused to generate the edge visual channels of each target image. The texture recognition unit generates texture channels for each target image based on the edge visual channel using the directional gradient histogram algorithm. The texture channels include the gradient direction and amplitude distribution of the target image.

[0007] Preferably, the preprocessing module obtains image light field particles for each target image, specifically including: S1: Use the Sobel operator to calculate the gradient of the grayscale image in the x and y directions; S2: For any pixel, take its... The neighborhood window is used to calculate the gradient statistics within the window and the mean of the gradient statistics in the neighborhood window. The gradient statistics include the square of the gradient in the x-direction, the square of the gradient in the y-direction, and the product of the gradients in the x and y directions. S3: Calculate the covariance matrix using the mean of the gradient statistics, and calculate the eigenvalues ​​of the covariance matrix. Use the Shi-Tomasi corner point determination criterion to filter out pixels with values ​​greater than a set threshold and mark them as corner points. The multiple corner points are used to mark the contours of each target image. S4: Sort the feature values ​​corresponding to multiple pixels that meet the requirements according to their numerical values, and use the corner points as the image light field particles of the corresponding target image.

[0008] Preferably, the motion trend data includes the displacement direction, velocity, and acceleration of the image light field particles. Determining the displacement direction includes: using a mean-shift algorithm to track the position changes of the image light field particles in consecutive frames of images, calculating the probability density distribution of pixel grayscale in the neighborhood of the image light field particles, iteratively finding the density maxima, and determining the displacement direction of the image light field particles using the density maxima.

[0009] Preferably, the optical flow trajectory mode execution process includes: A. Construct Gaussian pyramids from two consecutive frames of each target image, calculate optical flow starting from the top low-resolution image, and pass the calculation results layer by layer to the bottom high-resolution image. B. During the window matching process of the LK algorithm, the gradient magnitude of the HOG texture channel output by the texture recognition module is used as a constraint to perform displacement trajectory analysis and accumulate the optical flow vectors of consecutive frames to form a preliminary optical flow trajectory. C. The RANSAC algorithm is used to remove outliers in the initial optical flow trajectory, and after smoothing and removing high-frequency jitter, the optical flow trajectory of each target image is output.

[0010] Preferably, in step B, the optical flow vector constructed by the LK algorithm The formula for solving the system of equations is defined as follows: Where M is used to describe the grayscale change trend of the set pixel in the x and y directions, and N is used to describe the amount of grayscale change required for optical flow compensation of the set pixel.

[0011] Preferably, in step B, the gradient magnitude serves as a constraint condition, including: The gradient magnitude h is used as a weight value to weight M and N, where: The formula for calculating h is: ,in, They are respectively exist The directional gradients at x and y points; The weighted calculation formula is: ; ; Substituting the weighted matrix into the set overdetermined equations, we obtain the weighted overdetermined equations. The optimal optical flow vector is then calculated using the weighted least squares method.

[0012] Preferably, in step B, the formula for calculating the cumulative optical flow vectors of consecutive frames is: ; Among them, among them, Let k be the coordinates of the image light field particles at the initial moment, and k be the current pyramid level. This refers to the horizontal optical flow vector components of the image light field particles at the current pyramid level k, used to describe the amplitude and direction of motion in the horizontal direction. For the current pyramid level k, the calibrated image light field particles are the optical flow vector components in the vertical direction, used to describe the amplitude and direction of motion in the vertical direction.

[0013] Preferably, before performing edge visual feature analysis on each target image, the texture recognition module further includes illumination normalization processing. The normalization processing includes: performing illumination separation on the target image using the Retinex algorithm, decomposing the target image into reflection components and illumination components, suppressing global brightness fluctuations of the illumination components through logarithmic transformation, and then performing adaptive histogram equalization on the reflection components to enhance the local texture contrast of the reflection components; based on the processed reflection components, performing edge visual feature analysis to generate multiple edge visual channels.

[0014] An image recognition sensor is provided, applicable to an image recognition system for a camera as described in claim 1, wherein the image recognition sensor is disposed in the image recognition module and is used to realize the image information acquisition function of the image recognition module.

[0015] The beneficial effects of this invention are as follows: This solution achieves real-time processing at high frame rates by combining FPGA hardware acceleration with redundancy reduction through particle screening. This enables the image recognition system to stably handle the entire high-speed camera processing chain, meeting the latency-free tracking requirements of high-speed moving targets. Secondly, it significantly improves the stability of feature extraction for high-speed moving targets. The texture channel fusion of multiple edge channels improves the edge integrity rate in high-speed motion blur scenes and reduces the tracking loss rate of light field particles, ensuring that even if the target is locally blurred due to high speed, complete and stable features can still be extracted. Thirdly, it significantly improves the optical flow and tracking accuracy between high-speed frames. The designed dual-constraint optical flow mechanism reduces the optical flow estimation error of high-speed moving targets and reduces the inter-frame position deviation of camera tracking, achieving accurate positioning of high-speed targets. This solves the problems of image accuracy degradation and tracking breakage caused by high-speed target movement in high-speed camera image recognition. Attached Figure Description

[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0017] Figure 1 This is a system block diagram of the present invention; Figure 2 This is a flowchart illustrating the steps of the preprocessing module of the present invention for calculating image light field particles. Detailed Implementation

[0018] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0019] Please see Figures 1-2This embodiment provides an image recognition system for cameras, specifically for high-speed cameras. The system includes an image recognition module, a preprocessing module, a texture recognition module, an optical flow trajectory module, and a camera tracking module, which are sequentially and communicatively connected. Specifically: The image recognition module is used to acquire image information using an image recognition sensor and extract at least one target object from the acquired image information. The image algorithm based on FPGA performs pixel scanning on the image information to obtain a grayscale image. The image information includes continuous frame images. The preprocessing module uses a corner detection algorithm based on grayscale images to obtain the corner points of each target object. It then filters the corner points of each target image to obtain the image light field particles for each target image. Specifically, this includes: S1: Use the Sobel operator to calculate the gradient of the grayscale image in the x and y directions; S2: For any pixel, take its... The neighborhood window is used to calculate the gradient statistics within the window and the mean of the gradient statistics in the neighborhood window. The gradient statistics include the square of the gradient in the x direction, the square of the gradient in the y direction, and the product of the gradients in the x and y directions. S3: Calculate the covariance matrix using the mean of the gradient statistics, and calculate the eigenvalues ​​of the covariance matrix. Use the Shi-Tomasi corner point determination criterion to filter out pixels with values ​​greater than a set threshold and mark them as corner points. Multiple corner points are used to mark the contours of each target image. S4: Sort the feature values ​​corresponding to multiple pixels that meet the requirements according to their numerical values, and use the corner points as the image light field particles of the corresponding target image. Because the traditional Harris algorithm is not good at distinguishing between edge points and corner points, the selected feature particles contain a large number of low-contrast points or edge points, which leads to large errors in subsequent motion trend analysis and easy tracking drift. Corner points whose brightness does not reach the threshold are filtered out by setting corner point judgment criteria. The preprocessing module is also used to track the position changes of each image light field particle in consecutive frame images to obtain image light field particle motion trend data. The motion trend data includes the displacement direction, velocity, and acceleration of the image light field particles. Determining the displacement direction includes: using a mean drift algorithm to track the position changes of the image light field particles in consecutive frame images, calculating the probability density distribution of pixel gray levels in the neighborhood of the image light field particle, iteratively finding the density maxima, and using the density maxima to determine the displacement direction of the image light field particles.

[0020] The texture recognition module is used to perform edge visual feature analysis on each target image, generate multiple edge visual channels, and generate texture channels for each target image based on the edge visual channels. Specifically: The texture recognition module includes an edge recognition unit and a texture recognition unit; The edge recognition unit uses the multi-scale Canny edge detection algorithm to perform edge visual feature analysis: by setting high and low threshold gradients, the edge contours of each target object are extracted in multiple scale spaces, and the multi-scale edge information is fused to generate the edge visual channels of each target image. The texture recognition unit, based on the edge visual channel, uses the directional gradient histogram algorithm to generate texture channels for each target image. The texture channels include the gradient direction and magnitude distribution of the target image. The traditional pyramid Lucas-Kanade (LK) algorithm relies on image grayscale information as the only constraint, which is easily affected by changes in illumination, low-texture areas, dynamic noise or occlusion, leading to deviations in optical flow estimation. Furthermore, it only addresses large displacement errors through pyramids and does not effectively process abnormal optical flow vectors after calculation, resulting in broken or jittery trajectories. This further exacerbates the impact of high-speed camera movement on image recognition accuracy.

[0021] The optical flow trajectory module, based on the HOG texture channel output by the texture recognition module and the light field feature particle motion trend data output by the preprocessing module, analyzes the target's displacement trajectory in consecutive frames through a multi-stage algorithm, ultimately outputting a smooth and accurate optical flow displacement trajectory. Its execution process includes: A. Construct Gaussian pyramids from two consecutive frames (the current frame and the previous frame) of each target image. Calculate optical flow starting from the top low-resolution image and progressively pass the calculation results to the bottom high-resolution images, including: A1: Constructing the Gaussian Pyramid: Set the number of pyramid layers, and dynamically set the number of layers L according to the target's movement speed (usually 3-5 layers, and higher layers for fast-moving targets). The 0th layer is the original image, and the lth layer (l≥1) is the downsampled result of the l−1th layer. Apply Gaussian filtering to the (l-1)th layer image. Gaussian filtering is performed to suppress high-frequency noise; the filter kernel G is... The Gaussian matrix, the filtered image is calibrated as follows , The convolution operator performs downsampling, sampling the filtered image in alternating rows and columns to generate the l-th layer image. The size is Half of it, meaning the length and width of the image are both halved; A2: Calculate optical flow: Set the initial optical flow, and set the initial optical flow value of the top layer (l=L-1) to... ; Perform optical flow iterations layer by layer, and for each layer l (L-1 to 0) of the pyramid, perform the following operations: Optical flow results for layer l+1 Upsample (multiply by 2) to obtain the initial optical flow for the l-th layer; LK iterative optimization is performed based on the assumption of constant brightness, that is, the brightness of the same image light field feature particle remains unchanged in consecutive frames. For one image light field feature particle in the l-th layer image, it is labeled as of Using a neighborhood window, the optical flow calculation equation is derived through Taylor expansion: ; in, for exist The x and y gradients at a given location are provided based on the HOG texture channels. and Let t be the optical flow to be calculated, i.e., displacement, and t be the current t-th frame of the image.

[0022] B. During the window matching process of the LK algorithm, the gradient magnitude of the HOG texture channel output by the texture recognition module is used as a constraint to perform displacement trajectory analysis. The optical flow vectors of consecutive frames are accumulated to form a preliminary optical flow trajectory, including: B1: Calculate and process the overdetermined system of equations. For each of the selected n pixels within the window, a system of equations is generated. The expression for this system is: M and N are overdetermined equations based on the "constant brightness assumption" used to solve for the optical flow vector. M represents the displacement of the target image in consecutive frames, n is the number of pixels in the neighborhood window, and the M matrix is ​​a set of directional features of gray-level changes in local regions. The two elements in each row quantify the intensity of gray-level changes of the pixel in the horizontal and vertical directions, respectively, and are used to establish the optical flow vector. How to influence the mathematical relationship of grayscale changes? Each element of matrix N corresponds to the grayscale difference of a pixel in two consecutive frames within the window. It is used to describe the amount of grayscale change that needs to be compensated by optical flow. N is used to calculate the "constant brightness assumption" based on optical flow, that is, the grayscale of the same target point should remain unchanged in consecutive frames.

[0023] B2: HOG texture constraint optimization is performed by introducing the gradient magnitude h of the HOG texture channel as a weight value, and weighting M and N is applied. The formula for calculating h is: ,in, They are respectively exist The directional gradients at x and y points; The weighted calculation formula is: ; ; B3: Substitute the weighted weight matrix into the overdetermined equations of B1 to obtain the weighted overdetermined equations. Calculate the optimal optical flow vector using the weighted least squares method. B4: For the calibrated image light field particles, the optimal optical flow vectors from frame t=1 to t=T are accumulated one by one. Then, the series of optimal optical flow vectors obtained are accumulated onto the initial pixel coordinates of the calibrated image light field particles to obtain the preliminary optical flow trajectory. The cumulative calculation of continuous optical flow vectors is as follows: ,in,

[0024] Let k be the coordinates of the image light field particles at the initial moment, and k be the current pyramid level. This refers to the optical flow vector components of the calibrated image light field particles in the horizontal direction (usually the x-axis of the image, with the positive direction set to the horizontal right of the image) at the current pyramid level k, used to describe the amplitude and direction of motion in the vertical direction. This refers to the optical flow vector component of the calibrated image light field particles in the vertical direction (usually the y-axis direction of the image) at the current pyramid level k. It is used to describe the amplitude and direction of motion in the vertical direction, with the positive direction set to vertically downwards from the image.

[0025] The initial formula for calculating the optical flow trajectory is used to represent the position of the calibrated image light field particles starting from their initial positions. For each frame from the first frame to the t-th frame, at the current pyramid level k, the horizontal and vertical optical flow components are accumulated and summed to obtain the position of the calibrated image light field particles at time t. By accumulating such optical flow vectors, it is possible to track the motion trajectory of a target in consecutive frames of images, providing a basis for subsequent motion compensation, target localization, and other tasks.

[0026] C. Outliers in the initial optical flow trajectories are removed using the RANSAC algorithm, followed by smoothing to remove high-frequency jitter before outputting the optical flow trajectories of each target image. This specifically includes: The optical flow vectors of consecutive frames are accumulated to form a preliminary trajectory. Abnormal optical flow caused by noise or background interference is removed by the RANSAC algorithm, i.e., abnormal points are removed. The optical flow trajectory with abnormal points removed is smoothed by a window size of 3-5 to remove high-frequency jitter, and the final optical flow displacement trajectory is obtained.

[0027] Traditional pyramid LK algorithms typically involve calculating optical flow vectors layer by layer within the pyramid, directly outputting the optical flow vector. However, they lack a step to handle anomalous optical flow. When occlusion exists in the scene (e.g., abrupt changes in optical flow vectors due to occlusion) or dynamic noise (e.g., spurious optical flow caused by motion blur), the directly output optical flow vector will contain numerous outliers, leading to distorted final trajectories. The RANSAC algorithm is used to remove outliers from the initial optical flow trajectory. RANSAC (Random Sample Consensus) uses random sampling combined with model validation to select optical flow vectors (interior points) that conform to real motion patterns, while eliminating outliers (outterior points, such as erroneous optical flow caused by occlusion or noise). This step addresses the shortcoming of traditional pyramid LK algorithms, which only calculate without filtering, reducing trajectory errors at the source. Furthermore, the effective optical flow vectors filtered by RANSAC are integrated over time (i.e., the cumulative displacement increment across consecutive frames), and a smooth trajectory is obtained through curve fitting (e.g., least squares method). This solves the problem of discontinuous trajectories after concatenating discrete optical flow vectors in tracking high-speed moving targets, improving image recognition accuracy.

[0028] The camera tracking module is used to perform subsequent localization and tracking of light field particles in each image based on the optical flow displacement trajectory using image tracking algorithms. The camera here includes multiple cameras to cover the tracking and scanning range required by this solution. Specifically, it includes: The camera tracking module receives the optical flow displacement trajectory of each image light field particle output by the optical flow trajectory module. The optical flow displacement trajectory includes the historical position coordinates, displacement vector and motion trend parameters of the image light field particle in consecutive frames. At the same time, it synchronously acquires the latest frame image captured by the camera as the basic input data for tracking. Subsequently, the tracking state of each image light field particle is initialized: based on its historical optical flow displacement trajectory, motion feature parameters such as average velocity, acceleration, and motion direction angle are extracted to construct the particle's motion model. A uniform linear model or a uniform acceleration model is adopted, which is adaptively selected according to the degree of trajectory fluctuation. The feature template of the particle is recorded. The feature template contains the gray-scale neighborhood information, HOG texture features, and corner response values ​​of the particle in the historical frame, which serve as the benchmark for subsequent matching. For the latest frame image, the module first predicts the candidate location region of the particle in the current frame based on the particle's motion model and the optical flow displacement trajectory of the previous frame. Within the candidate location area, the camera tracking module invokes the corner detection sub-function of the preprocessing module to re-detect local corners and extract their features, including grayscale distribution and texture gradient. These features are then matched with the particle's feature template using normalized similarity. If a corner has a matching score exceeding a threshold (usually set to 0.7), it is determined as the preliminary localization result of the particle in the current frame. For the preliminary positioning results, the camera tracking module verifies them by combining the continuity constraint of the optical flow trajectory: the deviation between the current positioning position and the position predicted by the motion model is calculated. If the deviation is less than a preset threshold, the position is confirmed as the final positioning of the particle. If the deviation is too large, a secondary search is initiated to expand the candidate area. The corner points are re-selected under the constraint of texture feature matching, combined with the texture channel output by the texture recognition module, until a positioning point that meets the trajectory continuity is found, or the particle is marked as temporarily lost. For particles marked as temporarily lost, the camera tracking module predicts their possible positions in the next 2-3 frames based on the trend of their historical optical flow displacement trajectories using linear interpolation or Kalman filtering, and continues to search in the corresponding area. If they are not found for 3 consecutive frames, they are removed from the tracking list, and the preprocessing module is triggered to re-detect corner points within the target object range and supplement new light field particles to maintain tracking integrity.

[0029] Finally, the camera tracking module summarizes the current positioning results of all particles, calculates the overall motion parameters of the target object (such as center position, moving speed, and rotation angle) by weighted averaging, with the weight being the feature matching degree of the particles, and outputs the positioning coordinates of each particle and the overall motion parameters of the target, providing data support for real-time camera adjustment (such as lens focusing and viewpoint rotation) or subsequent applications (such as target behavior analysis).

[0030] This solution achieves real-time processing at high frame rates by combining FPGA hardware acceleration with redundancy reduction through particle screening. This enables the image recognition system to stably handle the entire high-speed camera processing chain, meeting the latency-free tracking requirements of high-speed moving targets. Secondly, it significantly improves the stability of feature extraction for high-speed moving targets. The texture channel fusion of multiple edge channels improves the edge integrity rate in high-speed motion blur scenes and reduces the tracking loss rate of light field particles, ensuring that even if the target is locally blurred due to high speed, complete and stable features can still be extracted. Thirdly, it significantly improves the optical flow and tracking accuracy between high-speed frames. The designed dual-constraint optical flow mechanism reduces the optical flow estimation error of high-speed moving targets and reduces the inter-frame position deviation of camera tracking, achieving accurate positioning of high-speed targets. This solves the problems of image accuracy degradation and tracking breakage caused by high-speed target movement in high-speed camera image recognition.

[0031] Because changes in the lighting environment of the camera can cause uneven brightness distribution in the target image, such as direct strong light, shadow occlusion, and switching between indoor and outdoor lighting, the edge visual channel generated by the texture recognition module may produce "false edges" (such as misjudging a sudden change in lighting as an edge) or "edge breaks" (such as the real edge of the shadow area being covered up), which in turn reduces the accuracy of the optical flow trajectory module in analyzing the optical flow displacement trajectory based on the texture channel.

[0032] The texture recognition module also includes using the Retinex algorithm to separate the target image from its illumination, decomposing it into reflection and illumination components. Logarithmic transformation is used to suppress global brightness fluctuations in the illumination component, and adaptive histogram equalization is then performed on the reflection component to enhance its local texture contrast. Based on the processed reflection component, edge visual feature analysis is performed to generate multiple edge visual channels. This eliminates the interference of illumination changes on texture features at the source. Adaptive histogram equalization further enhances the texture details of the reflection component, ensuring that the edge visual channels only reflect the edge features of the target's inherent texture. This provides stable texture constraints for the optical flow trajectory module, avoiding optical flow trajectory deviations caused by illumination changes.

[0033] An image recognition sensor is provided, which is disposed in an image recognition module of an image recognition system applied to a camera as described above. The image recognition sensor is used to realize the image information acquisition function of the image recognition module.

[0034] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An image recognition system applied to a camera, characterized in that: include: An image recognition module is used to acquire image information and extract at least one target object from the acquired image information. An FPGA-based image algorithm performs pixel scanning on the image information to obtain a grayscale image. The image information includes continuous frame images. The preprocessing module uses a corner detection algorithm based on grayscale images to obtain the corner points of each target object, and filters the corner points of each target image to obtain the image light field particles of each target image. The position changes of each image light field particle in consecutive frames are tracked to obtain image light field particle motion trend data; The texture recognition module is used to perform edge visual feature analysis on each target image, generate multiple edge visual channels, and generate texture channels for each target image based on the edge visual channels. The optical flow trajectory module is used to perform optical flow displacement trajectory analysis on the motion trend data of light field particles based on the image texture channel, so as to obtain the optical flow displacement trajectory of each target image; The camera tracking module is used to perform subsequent positioning and tracking of each image light field particle based on the optical flow displacement trajectory using an image tracking algorithm.

2. The image recognition system applied to a camera according to claim 1, characterized in that: The texture recognition module includes an edge recognition unit and a texture recognition unit; The edge recognition unit uses the multi-scale Canny edge detection algorithm to perform edge visual feature analysis: by setting high and low threshold gradients, the edge contours of each target object are extracted in multiple scale spaces, and the multi-scale edge information is fused to generate the edge visual channels of each target image. The texture recognition unit generates texture channels for each target image based on the edge visual channel using the directional gradient histogram algorithm. The texture channels include the gradient direction and amplitude distribution of the target image.

3. The image recognition system applied to a camera according to claim 1, characterized in that: The preprocessing module obtains the image light field particles of each target image, specifically including: S1: Use the Sobel operator to calculate the gradient of the grayscale image in the x and y directions; S2: For any pixel, take its... The neighborhood window is used to calculate the gradient statistics within the window and the mean of the gradient statistics in the neighborhood window. The gradient statistics include the square of the gradient in the x-direction, the square of the gradient in the y-direction, and the product of the gradients in the x and y directions. S3: Calculate the covariance matrix using the mean of the gradient statistics, and calculate the eigenvalues ​​of the covariance matrix. Use the Shi-Tomasi corner point determination criterion to filter out pixels with values ​​greater than a set threshold and mark them as corner points. The multiple corner points are used to mark the contours of each target image. S4: Sort the feature values ​​corresponding to multiple pixels that meet the requirements according to their numerical values, and use the corner points as the image light field particles of the corresponding target image.

4. The image recognition system applied to a camera according to claim 1, characterized in that: The motion trend data includes the displacement direction, velocity, and acceleration of the image light field particles. Determining the displacement direction includes: using a mean-shift algorithm to track the position changes of the image light field particles in consecutive frames of images, calculating the probability density distribution of pixel grayscale in the neighborhood of the image light field particles, iteratively finding the density maxima, and using the density maxima to determine the displacement direction of the image light field particles.

5. An image recognition system for a camera according to claim 1, characterized in that: The optical flow trajectory mode execution process includes: A. Construct Gaussian pyramids from two consecutive frames of each target image, calculate optical flow starting from the top low-resolution image, and pass the calculation results layer by layer to the bottom high-resolution image. B. During the window matching process of the LK algorithm, the gradient magnitude of the HOG texture channel output by the texture recognition module is used as a constraint to perform displacement trajectory analysis and accumulate the optical flow vectors of consecutive frames to form a preliminary optical flow trajectory. C. The RANSAC algorithm is used to remove outliers in the initial optical flow trajectory, and after smoothing and removing high-frequency jitter, the optical flow trajectory of each target image is output.

6. An image recognition system for a camera according to claim 5, characterized in that: In step B, the optical flow vector constructed by the LK algorithm The formula for solving the system of equations is defined as follows: Where M is used to describe the grayscale change trend of the set pixel in the x and y directions, and N is used to describe the amount of grayscale change required for optical flow compensation of the set pixel.

7. An image recognition system for a camera according to claim 6, characterized in that: In step B, the gradient magnitude serves as a constraint condition, including: The gradient magnitude h is used as a weight value to weight M and N, where: The formula for calculating h is: ,in, They are respectively exist The directional gradients at x and y points; The weighted calculation formula is: ； ； Substituting the weighted matrix into the set overdetermined equations, we obtain the weighted overdetermined equations. The optimal optical flow vector is then calculated using the weighted least squares method.

8. An image recognition system for a camera according to claim 6, characterized in that: In step B, the formula for calculating the cumulative optical flow vectors of consecutive frames is: ,in, Let k be the coordinates of the image light field particles at the initial moment, and k be the current pyramid level. This refers to the horizontal optical flow vector components of the image light field particles at the current pyramid level k, used to describe the amplitude and direction of motion in the horizontal direction. For the current pyramid level k, the calibrated image light field particles are the optical flow vector components in the vertical direction, used to describe the amplitude and direction of motion in the vertical direction.

9. An image recognition system for a camera according to claim 1, characterized in that: Before performing edge visual feature analysis on each target image, the texture recognition module also includes illumination normalization processing. The normalization processing includes: using the Retinex algorithm to separate the illumination of the target image into reflection and illumination components, suppressing global brightness fluctuations of the illumination component through logarithmic transformation, and then performing adaptive histogram equalization on the reflection component to enhance the local texture contrast of the reflection component; based on the processed reflection component, edge visual feature analysis is performed to generate multiple edge visual channels.

10. An image recognition sensor, characterized in that: The image recognition system applicable to a camera as described in claim 1, wherein the image recognition sensor is disposed in the image recognition module, and the image recognition sensor is used to realize the image information acquisition function of the image recognition module.

Images