An image recognition system and image recognition sensor applied to a camera

Abstract

The application belongs to the technical field of image processing, and particularly relates to an image recognition system applied to a camera and an image recognition sensor, the system comprising an image recognition module, a preprocessing module, a texture recognition module, an optical flow trajectory module and a camera tracking module which are sequentially and communicatively connected, real-time processing under high frame frequency is realized, FPGA hardware acceleration is combined with redundancy reduction of particle screening, the image recognition system can stably process a full link of a high-speed camera, and the no-delay tracking demand of a high-speed moving target is met; the edge integrity rate under a high-speed motion blur scene is improved through a texture channel of multi-edge channel fusion, and the tracking loss rate of an optical field particle is reduced; the optical flow estimation error of a high-speed moving target is reduced through a designed double-constraint optical flow mechanism, the interframe position deviation of camera tracking is reduced, accurate positioning of a high-speed target is realized, and the problems of image precision reduction and tracking rupture caused by high-speed movement of a target in high-speed camera image recognition are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition, and particularly relates to an image recognition system applied to a camera and an image recognition sensor. BACKGROUND

[0002] The importance of high-speed camera image recognition systems in modern technology applications is self-evident. Such systems are widely used in many fields, such as industrial automation, intelligent transportation systems, aerospace, military reconnaissance, etc. Accurate and rapid recognition of high-speed moving targets is a key technology for the shooting effect of high-speed cameras.

[0003] However, the image processing system of the high-speed camera in some existing technologies faces many challenges when dealing with the recognition of high-speed moving targets. Because the target moves at a very high speed, traditional processing methods often cannot process and recognize images in real time, which leads to a lack of compensation for the high-speed motion characteristics of the target in the recognition algorithm when processing continuous high-speed images. The inter-frame position deviation of the camera tracking the target caused by the high-speed movement of the target further causes local blur of the image, thereby affecting the accuracy of image recognition. SUMMARY

[0004] To solve the above problems in the prior art, the application provides an image recognition system applied to a camera and an image recognition sensor, which solves the problem of local blur of images caused by the inter-frame position deviation of the camera tracking the target caused by the high-speed movement of the target when the image processing system of the high-speed camera deals with the recognition of high-speed moving targets.

[0005] The purpose of the application can be achieved by the following technical scheme: an image recognition system applied to a camera, comprising:

[0006] An image recognition module is configured to collect image information and extract at least one target object from the collected image information, perform pixel scanning on the image information based on an FPGA-based image algorithm to obtain a gray-scale image, and the image information comprises continuous frame images;

[0007] A preprocessing module is configured to obtain corner points of each target object based on the gray-scale image using a corner detection algorithm, and screen the corner points of each target image respectively to obtain image light field particles of each target image;

[0008] The position changes of each image light field particle in the continuous frame images are tracked respectively, so as to obtain image light field particle motion trend data;

[0009] A texture recognition module is configured to perform edge visual feature analysis on each target image, generate a plurality of edge visual channels, and generate a texture channel of each target image based on the edge visual channels;

[0010] an optical flow trajectory module, configured to perform optical flow displacement trajectory analysis on the light field particle motion trend data based on the image texture channel to obtain optical flow displacement trajectories of each target image;

[0011] a camera tracking module, configured to perform subsequent positioning tracking on each image light field particle based on each optical flow displacement trajectory using an image tracking algorithm.

[0012] Preferably, the texture recognition module comprises an edge recognition unit and a texture recognition unit.

[0013] The edge recognition unit performs edge visual feature analysis using a multi-scale Canny edge detection algorithm: by setting high and low threshold gradients, edge contours of each target object are extracted in a plurality of scale spaces respectively, and multi-scale edge information is fused to generate an edge visual channel of each target image.

[0014] The texture recognition unit generates a texture channel of each target image using a histogram of oriented gradients algorithm based on the edge visual channel, and the texture channel comprises gradient direction and amplitude distribution of the target image.

[0015] Preferably, the preprocessing module obtains image light field particles of each target image, and specifically comprises:

[0016] S1: using a Sobel operator to calculate the gradients of a gray image in x and y directions;

[0017] S2: for any pixel point, taking a neighborhood window of the pixel point, calculating gradient statistics in the window and calculating the mean value of the gradient statistics in the neighborhood window, the gradient statistics including the square of the x direction gradient, the square of the y direction gradient, and the product of the x and y direction gradients; S3: calculating a covariance matrix with the mean value of the gradient statistics, and calculating eigenvalues of the covariance matrix, using a Shi-Tomasi corner determination criterion to screen pixel points with eigenvalues greater than a set threshold and marking them as corner points, and a plurality of the corner points are used to mark the contours of each target image respectively;

[0018] S4: sorting the eigenvalues corresponding to a plurality of pixel points meeting the requirements according to the numerical value, and taking the corner points as the image light field particles of the corresponding target image.

[0019]

[0020] ​Preferably, the motion trend data comprises displacement direction, velocity and acceleration of image light field particles, wherein the displacement direction is determined by tracking the position change of image light field particles in consecutive frame images using a mean shift algorithm, and finding the density maximum point by calculating the probability density distribution of pixel gray scale in the neighborhood of image light field particles, and determining the displacement direction of image light field particles by the density maximum point.

[0021] Preferably, the light flow trajectory module comprises:

[0022] A. constructing two consecutive frame images of each target image into a Gaussian pyramid, starting from the top layer low-resolution image to calculate the optical flow, and transferring the calculation result to the bottom layer high-resolution image layer by layer;

[0023] B. in the window matching process of the LK algorithm, taking the gradient amplitude of the HOG texture channel output by the texture recognition module as a constraint condition to analyze the displacement trajectory, and accumulating the optical flow vectors of consecutive frames to form a preliminary optical flow trajectory;

[0024] C. removing outliers in the preliminary optical flow trajectory by RANSAC algorithm, performing smoothing processing, and outputting the optical flow trajectory of each target image after removing high-frequency jitter.

[0025] Preferably, in step B, the optical flow vector solved by the LK algorithm is calibrated as: , wherein M is used to describe the gray scale change trend of the set pixel in the x and y directions, and N is used to describe the gray scale change amount of the set pixel that needs to be compensated by the optical flow.

[0026] Preferably, in step B, the gradient amplitude as a constraint condition comprises:

[0027] The gradient amplitude h is taken as a weight value to weight M and N, wherein:

[0028] The calculation formula of h is: , wherein are the directional gradients of x and y at .

[0029] The weighted calculation formula is:

[0030] .

[0031] .

[0032] The weight matrix after weighting is substituted into the set overdetermined equation set to obtain a weighted overdetermined equation set, and the optimal optical flow vector is calculated by the weighted least square method. ​

[0033] Preferably, in step B, the calculation formula of the optical flow vector accumulation of the continuous frames is:

[0034]

[0035] wherein, wherein, is the position coordinate of the calibrated image light field particle at the initial moment, k is the current pyramid level number, is the horizontal direction optical flow vector component of the calibrated image light field particle at the current pyramid level k, used to describe the motion amplitude and direction in the horizontal direction, is the vertical direction optical flow vector component of the calibrated image light field particle at the current pyramid level k, used to describe the motion amplitude and direction in the vertical direction.

[0036] Preferably, before the edge visual feature analysis of each target image by the texture recognition module, the target image further comprises a light normalization processing, and the normalization processing comprises: performing light separation on the target image by using a Retinex algorithm, decomposing the target image into a reflection component and a light component, suppressing the global brightness fluctuation of the light component through logarithmic transformation, and then performing adaptive histogram equalization on the reflection component to enhance the local texture contrast of the reflection component; based on the processed reflection component, edge visual feature analysis is performed to generate a plurality of edge visual channels.

[0037] An image recognition sensor suitable for the image recognition system applied to the camera, the image recognition sensor is arranged in the image recognition module, and the image recognition sensor is used to realize the image information acquisition function of the image recognition module.

[0038] The beneficial effects of the present application are:

[0039] The present application realizes real-time processing under high frame frequency, combines FPGA hardware acceleration and redundancy reduction of particle selection, so that the image recognition system can stably process the full link of the high-speed camera, meet the non-delay tracking demand of the high-speed moving target, greatly improve the feature extraction stability of the high-speed moving target, improve the edge integrity rate in the high-speed motion blur scene through the texture channel of the multi-edge channel fusion, reduce the tracking loss rate of the light field particle, and ensure that even if the target is locally blurred due to high speed, complete and stable features can still be extracted;The third is to significantly improve the optical flow and tracking accuracy between high-speed frames, reduce the optical flow estimation error of the high-speed moving target through the designed double-constraint optical flow mechanism, reduce the inter-frame position deviation of the camera tracking, realize accurate positioning of the high-speed target, and solve the problems of image accuracy decline and tracking rupture caused by high-speed movement of the target in high-speed camera image recognition. BRIEF DESCRIPTION OF DRAWINGS

[0040] ​To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0041] Fig. 1 This is a system block diagram of the present invention;

[0042] Fig. 2 This is a flowchart illustrating the steps of the preprocessing module of the present invention for calculating image light field particles. Detailed Implementation

[0043] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0044] Please see Figs. 1-2 This embodiment provides an image recognition system for cameras, specifically for high-speed cameras. The system includes an image recognition module, a preprocessing module, a texture recognition module, an optical flow trajectory module, and a camera tracking module, which are sequentially and communicatively connected. Specifically:

[0045] The image recognition module is used to acquire image information using an image recognition sensor and extract at least one target object from the acquired image information. The image algorithm based on FPGA performs pixel scanning on the image information to obtain a grayscale image. The image information includes continuous frame images.

[0046] The preprocessing module uses a corner detection algorithm based on grayscale images to obtain the corner points of each target object. It then filters the corner points of each target image to obtain the image light field particles for each target image. Specifically, this includes:

[0047] S1: Use the Sobel operator to calculate the gradient of the grayscale image in the x and y directions;

[0048] S2: For any pixel, take its... The neighborhood window is used to calculate the gradient statistics within the window and the mean of the gradient statistics in the neighborhood window. The gradient statistics include the square of the gradient in the x direction, the square of the gradient in the y direction, and the product of the gradients in the x and y directions.

[0049] S3: Calculate the covariance matrix using the mean of the gradient statistics, and calculate the eigenvalues ​​of the covariance matrix. Use the Shi-Tomasi corner point determination criterion to filter out pixels with values ​​greater than a set threshold and mark them as corner points. Multiple corner points are used to mark the contours of each target image.

[0050] S4: Sort the feature values ​​corresponding to multiple pixels that meet the requirements according to their numerical values, and use the corner points as the image light field particles of the corresponding target image.

[0051] Since the traditional Harris algorithm has insufficient distinction between edge points and corner points, the screened feature particles contain a large number of low-contrast points or edge points, leading to large errors in subsequent motion trend analysis, and easy tracking drift, and the corner points whose brightness does not reach the threshold are screened out through the set corner point determination criterion;

[0052] The preprocessing module is also used to track the position changes of each image light field particle in the continuous frame images respectively, so as to obtain image light field particle motion trend data, and the motion trend data includes the displacement direction, speed and acceleration of the image light field particle, wherein the displacement direction includes: using the mean shift algorithm to track the position changes of the image light field particle in the continuous frame images, finding the density maximum value point by calculating the probability density distribution of the pixel gray scale in the neighborhood of the image light field particle, and determining the displacement direction of the image light field particle by the density maximum value point.

[0053] The texture recognition module is used for edge visual feature analysis of each target image, generates a plurality of edge visual channels, and generates a texture channel of each target image based on the edge visual channel, and specifically:

[0054] The texture recognition module includes an edge recognition unit and a texture recognition unit;

[0055] The edge recognition unit performs edge visual feature analysis by using a multi-scale Canny edge detection algorithm: by setting high and low threshold gradients, the edge contours of each target object are extracted in a plurality of scale spaces respectively, and the multi-scale edge information is fused to generate an edge visual channel of each target image;

[0056] The texture recognition unit generates a texture channel of each target image based on the edge visual channel by using a histogram of oriented gradients algorithm, and the texture channel includes the gradient direction and amplitude distribution of the target image;

[0057] The traditional pyramid Lucas-Kanade (LK) algorithm relies on image gray scale information as the only constraint, is easily affected by light changes, low texture areas, dynamic noise or occlusion, and leads to flow estimation deviation; and only large displacement errors are solved by the pyramid, and the abnormal optical flow vectors after calculation are not effectively processed, so that the final trajectory is prone to breakage or jitter, and the influence of high-speed movement of the high-speed camera on image recognition accuracy is further intensified.

[0058] The optical flow trajectory module is based on the HOG texture channel output by the texture recognition module and the light field feature particle motion trend data output by the preprocessing module, analyzes the displacement trajectory of the target in the continuous frames through a multi-stage algorithm, and finally outputs a smooth and accurate optical flow displacement trajectory, and the execution process includes:

[0059] A, the two consecutive frames of each target image (current frame and previous frame) are respectively constructed into a Gaussian pyramid, starting from the top layer of low resolution image to calculate the optical flow, and the calculation result is transmitted to the bottom layer of high resolution image layer by layer, including:

[0060] A1: Construct a Gaussian pyramid:

[0061] Set the number of pyramid layers, dynamically set the number of layers L according to the target motion speed (usually 3-5 layers, faster motion target takes higher number of layers), the 0th layer is the original image, and the lth layer (l≥1) is the down-sampling result of the (l-1)th layer;

[0062] Perform Gaussian filtering on the (l-1)th layer image Perform Gaussian filtering to suppress high-frequency noise, and the filter kernel G is Gaussian matrix, the filtered image is denoted as , is a convolution operator; down-sampling is performed on the filtered image to sample every other row and column, generating the lth layer image , the size is half, that is, the length and width of the image are halved;

[0063] A2: Calculate the optical flow:

[0064] Set the initial optical flow, and set the initial value of the optical flow of the top layer (l=L-1) as ;

[0065] Perform optical flow iteration layer by layer, for each layer l (L-1 to 0) on the pyramid, perform the following operations:

[0066] Up-sample (multiply by 2) the optical flow result of the (l+1)th layer as the initial optical flow of the lth layer;

[0067] Perform LK iterative optimization based on the assumption of constant brightness, that is, the brightness of the same image light field feature particle does not change in consecutive frames. For one of the image light field feature particles in the lth layer image, denote the neighborhood window of as , the optical flow calculation equation is derived through Taylor expansion:

[0068] ;

[0069] where, is the x, y direction gradient of at , which is provided according to the HOG texture channel, and is the optical flow to be calculated, i.e. displacement, t is the current tth frame image.

[0070] B. In the window matching process of the LK algorithm, the gradient amplitude of the HOG texture channel output by the texture recognition module is taken as a constraint condition, displacement trajectory analysis is performed, and the optical flow vectors of consecutive frames are accumulated to form a preliminary optical flow trajectory, including:

[0071] B1: Calculate the overdetermined equation set and calculate it through the overdetermined equation set, wherein, for n pixels in the selected window, an equation set is generated, and the expression of the equation set is , wherein M and N are overdetermined equation sets constructed based on the "brightness constant assumption" for solving the optical flow vector, M is the displacement amount of the target image in consecutive frames, n is the number of pixels in the neighborhood window, the M matrix is a direction feature set of local region gray change, and two elements of each row quantize the gray change intensity of the pixel in the horizontal and vertical directions, which is used to establish the optical flow vector The mathematical relationship of how to affect the gray change, each element of the matrix N corresponds to the gray difference value of a pixel in the window in consecutive two frames, which is used to describe the gray change amount that needs to be compensated by the optical flow, and N is used to calculate the "brightness constant assumption" of the optical flow, that is, the gray of the same target point in consecutive frames should remain unchanged.

[0072] B2: Perform HOG texture constraint optimization by introducing the gradient amplitude h of the HOG texture channel as a weight value, and weight M and N, wherein the calculation formula of h is: , wherein , respectively The direction gradient at x, y in

[0073] The weighted calculation formula is:

[0074] ;

[0075] ;

[0076] B3: Substitute the weighted matrix after weighting into the overdetermined equation set of B1 to obtain a weighted overdetermined equation set, and calculate the optimal optical flow vector through the weighted least square method;

[0077] B4: For the calibrated image light field particles, the optimal optical flow vectors of t=1 to t=T frames are accumulated one by one, and then a series of optimal optical flow vectors obtained are accumulated to the initial pixel coordinates of the calibrated image light field particles to obtain a preliminary optical flow trajectory, and the accumulation calculation of consecutive optical flow vectors is:

[0078] , wherein

[0079] is the position coordinate of the calibrated image light field particle at the initial moment, k is the current pyramid level number, ​is a horizontal optical flow vector component of the calibrated image light field particle at the current pyramid level k, used to describe the motion amplitude and direction in the horizontal direction, and the positive direction is set as the right direction of the image. is a vertical optical flow vector component of the calibrated image light field particle at the current pyramid level k, used to describe the motion amplitude and direction in the vertical direction, and the positive direction is set as the downward direction of the image.

[0080] is a horizontal optical flow vector component of the calibrated image light field particle at the current pyramid level k, used to describe the motion amplitude and direction in the horizontal direction, and the positive direction is set as the right direction of the image. Through the accumulation of the optical flow vector, the motion trajectory of the target in the continuous frame image can be tracked, which provides a basis for subsequent motion compensation, target positioning, etc.

[0081] C. The RANSAC algorithm is used to remove outliers in the preliminary optical flow trajectory and perform smoothing processing, and the optical flow trajectory of each target image is output after removing high-frequency jitter, which specifically includes:

[0082] The optical flow vector of the continuous frame is accumulated to form a preliminary trajectory, and the RANSAC algorithm is used to remove abnormal optical flow caused by noise or background interference, that is, to remove abnormal points. The optical flow trajectory with the removed abnormal points is smoothed by using a window size of 3-5 to remove high-frequency jitter, and the final optical flow displacement trajectory is obtained.

[0083] The traditional pyramid LK algorithm usually has the process of pyramid layering calculation and directly outputs the optical flow vector, and lacks a processing link for abnormal optical flow. When there is occlusion (such as sudden change of the optical flow vector of the occluded part of the target) or dynamic noise (such as false optical flow caused by motion blur) in the scene, the directly output optical flow vector will contain a large number of abnormal values, resulting in distortion of the final trajectory. The RANSAC algorithm is used to remove the outliers in the preliminary optical flow trajectory: the core of RANSAC (Random Sample Consensus) is to filter out the optical flow vector (inlier) that conforms to the true motion law by random sampling combined with model verification, and remove the abnormal values (outlier) that do not conform, such as errors caused by occlusion and noise. This step fills the short board of the traditional pyramid LK algorithm which only calculates but does not filter, and reduces the trajectory error from the source. The effective optical flow vector filtered by RANSAC is integrated in the time dimension (i.e., the displacement increment of the continuous frame is accumulated), and a smooth trajectory is obtained through curve fitting (such as the least squares method). The problem of discontinuous trajectory after splicing of discrete optical flow vectors in the trajectory tracking of high-speed moving targets is solved, and the image recognition accuracy is improved.

[0084] The camera tracking module is used for subsequent positioning tracking of each image light field particle based on the optical flow displacement trajectory of each light flow displacement trajectory, wherein the camera includes a plurality of cameras that can cover the tracking and scanning range required by the scheme, and specifically includes:

[0085] The camera tracking module receives the optical flow displacement trajectory of each image light field particle output by the optical flow trajectory module, and the optical flow displacement trajectory includes the historical position coordinates, displacement vector and motion trend parameters of the image light field particle in the continuous frames. Meanwhile, the latest frame image captured by the camera is synchronously acquired as the basic input data for tracking.

[0086] Subsequently, the tracking state of each image light field particle is initialized: based on the historical optical flow displacement trajectory, the motion characteristic parameters such as average speed, acceleration and motion direction angle are extracted, the motion model of the particle is constructed, the uniform straight line model or the uniform acceleration model is adopted, and the trajectory fluctuation degree is adaptively selected, and the characteristic template of the particle is recorded, which includes the gray neighborhood information, HOG texture feature and corner response value of the particle in the historical frame, as the reference for subsequent matching.

[0087] For the latest frame image, the module first predicts the candidate position region of the particle in the current frame based on the motion model of the particle and the optical flow displacement trajectory of the last frame.

[0088] In the candidate position region, the camera tracking module calls the corner detection sub-function of the preprocessing module to re-detect local corners and extract the features of these corners, which include gray distribution and texture gradient, and performs normalized similarity matching with the characteristic template of the particle. If there is a corner with a matching degree exceeding a threshold value (usually set to 0.7), it is determined as the preliminary positioning result of the particle in the current frame.

[0089] For the preliminary positioning result, the camera tracking module verifies it in combination with the continuity constraint of the optical flow trajectory: the deviation between the current positioning position and the predicted position of the motion model is calculated, and if the deviation is less than a preset threshold value, the position is confirmed as the final positioning of the particle; if the deviation is too large, a secondary search is started, the candidate region is expanded, and the texture channel output by the texture recognition module is combined to re-select the corners under the constraint of texture feature matching, until a positioning point meeting the continuity of the trajectory is found, or the particle is marked as temporarily lost.

[0090] For the particle marked as temporarily lost, the camera tracking module predicts its possible positions in the subsequent 2-3 frames based on the trend of its historical optical flow displacement trajectory through linear interpolation or Kalman filtering, and continuously searches in the corresponding region; if it is not found for 3 consecutive frames, it is removed from the tracking list, and the preprocessing module is triggered to re-detect the corners in the target object range to supplement new light field particles to maintain the integrity of tracking.

[0091] Finally, the camera tracking module aggregates the current positioning results of all particles, calculates the overall motion parameters (such as center position, moving speed, rotation angle) of the target object by weighted average, and outputs the positioning coordinates of each particle and the overall motion parameters of the target, providing data support for real-time adjustment (such as lens focusing, view angle rotation) or subsequent application (such as target behavior analysis) of the camera.

[0092] The present scheme realizes real-time processing under high frame rate, combines FPGA hardware acceleration with redundancy reduction of particle selection, enables the image recognition system to stably process the full link of the high-speed camera, and meets the non-delay tracking requirements of high-speed moving targets; secondly, it greatly improves the stability of feature extraction of high-speed moving targets, the texture channel of multi-edge channel fusion improves the edge integrity rate in high-speed motion blur scenes, reduces the tracking loss rate of light field particles, and ensures that even if the target is locally blurred due to high speed, complete and stable features can still be extracted; thirdly, it significantly improves the optical flow and tracking accuracy between high-speed frames, reduces the optical flow estimation error of high-speed moving targets through the designed double-constraint optical flow mechanism, reduces the inter-frame position deviation of camera tracking, and realizes accurate positioning of high-speed targets, thereby solving the problems of image accuracy degradation and tracking breakage caused by high-speed target movement in high-speed camera image recognition.

[0093] Due to the change of light in the working environment of the camera, the brightness distribution of the target image will be uneven, such as direct sunlight, shadow shielding, indoor and outdoor light switching, which will cause the edge visual channel generated by the texture recognition module to have "pseudo-edges" (such as misjudging the edge at the light abrupt change) or "edge breakage" (such as the real edge being covered in the shadow area), thereby reducing the accuracy of the optical flow trajectory module in analyzing the optical flow displacement trajectory based on the texture channel.

[0094] The texture recognition module also includes using the Retinex algorithm to separate the light of the target image, decomposing the target image into a reflection component and a light component, suppressing the global brightness fluctuation of the light component through logarithmic transformation, and then performing adaptive histogram equalization on the reflection component to enhance the local texture contrast of the reflection component; based on the processed reflection component, edge visual feature analysis is performed to generate multiple edge visual channels. The interference of light changes on texture features is eliminated from the root; adaptive histogram equalization further enhances the texture details of the reflection component, ensuring that the edge visual channel only reflects the edge features of the inherent texture of the target, providing stable texture constraints for the optical flow trajectory module, and avoiding the displacement of the optical flow trajectory caused by light changes.

[0095] An image recognition sensor is provided in an image recognition module of an image recognition system as described above, and the image recognition sensor is used to realize the image information acquisition function of the image recognition module.

[0096] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content to obtain equivalent embodiments with equivalent changes, as long as the changes or modifications do not deviate from the technical solution of the present application. Any modification, change, equivalent change and modification of the above embodiments made according to the technical essence of the present application still belong to the scope of the technical solution of the present application.

Claims

1. An image recognition system applied to a camera, characterized in that: include: An image recognition module is used to acquire image information and extract at least one target object from the acquired image information. An FPGA-based image algorithm performs pixel scanning on the image information to obtain a grayscale image. The image information includes continuous frame images. The preprocessing module uses a corner detection algorithm based on grayscale images to obtain the corner points of each target object, and filters the corner points of each target image to obtain the image light field particles of each target image. The position changes of each image light field particle in consecutive frames are tracked to obtain image light field particle motion trend data; The texture recognition module is used to perform edge visual feature analysis on each target image, generate multiple edge visual channels, and generate texture channels for each target image based on the edge visual channels. The optical flow trajectory module is used to perform optical flow displacement trajectory analysis on the motion trend data of light field particles based on the image texture channel, so as to obtain the optical flow displacement trajectory of each target image; The camera tracking module is used to perform subsequent positioning and tracking of each image light field particle based on each of the optical flow displacement trajectories using an image tracking algorithm. The preprocessing module obtains the image light field particles of each target image, specifically including: S1: Use the Sobel operator to calculate the gradient of the grayscale image in the x and y directions; S2: For any pixel, take its... The neighborhood window is used to calculate the gradient statistics within the window and the mean of the gradient statistics in the neighborhood window. The gradient statistics include the square of the gradient in the x-direction, the square of the gradient in the y-direction, and the product of the gradients in the x and y directions. S3: Calculate the covariance matrix using the mean of the gradient statistics, and calculate the eigenvalues ​​of the covariance matrix. Use the Shi-Tomasi corner point determination criterion to filter out pixels with values ​​greater than a set threshold and mark them as corner points. The multiple corner points are used to mark the contours of each target image. S4: Sort the feature values ​​corresponding to multiple pixels that meet the requirements according to their numerical values, and use the corner points as the image light field particles of the corresponding target image. The optical flow trajectory mode execution process includes: A. Construct Gaussian pyramids from two consecutive frames of each target image, calculate optical flow starting from the top low-resolution image, and pass the calculation results layer by layer to the bottom high-resolution image. B. During the window matching process of the LK algorithm, the gradient magnitude of the HOG texture channel output by the texture recognition module is used as a constraint to perform displacement trajectory analysis and accumulate the optical flow vectors of consecutive frames to form a preliminary optical flow trajectory. C. The RANSAC algorithm is used to remove outliers in the initial optical flow trajectory, and after smoothing and removing high-frequency jitter, the optical flow trajectory of each target image is output.

2. The image recognition system applied to a camera according to claim 1, characterized in that: The texture recognition module includes an edge recognition unit and a texture recognition unit; The edge recognition unit uses the multi-scale Canny edge detection algorithm to perform edge visual feature analysis: by setting high and low threshold gradients, the edge contours of each target object are extracted in multiple scale spaces, and the multi-scale edge information is fused to generate the edge visual channels of each target image. The texture recognition unit generates texture channels for each target image based on the edge visual channel using the directional gradient histogram algorithm. The texture channels include the gradient direction and amplitude distribution of the target image.

3. The image recognition system applied to a camera according to claim 1, characterized in that: The motion trend data includes the displacement direction, velocity, and acceleration of the image light field particles. Determining the displacement direction includes: using a mean-shift algorithm to track the position changes of the image light field particles in consecutive frames of images, calculating the probability density distribution of pixel grayscale in the neighborhood of the image light field particles, iteratively finding the density maxima, and using the density maxima to determine the displacement direction of the image light field particles.

4. The image recognition system applied to a camera according to claim 1, characterized in that: In step B, the optical flow vector constructed by the LK algorithm The formula for solving the system of equations is defined as follows: Where M is used to describe the grayscale change trend of the set pixel in the x and y directions, and N is used to describe the amount of grayscale change required for optical flow compensation of the set pixel.

5. An image recognition system for a camera according to claim 4, characterized in that: In step B, the gradient magnitude serves as a constraint condition, including: The gradient magnitude h is used as a weight value to weight M and N, where: The formula for calculating h is: ,in, They are respectively exist The directional gradients at x and y points; The weighted calculation formula is: ； ； Substituting the weighted matrix into the set overdetermined equations, we obtain the weighted overdetermined equations. The optimal optical flow vector is then calculated using the weighted least squares method.

6. The image recognition system applied to a camera according to claim 1, characterized in that: In step B, the formula for calculating the cumulative optical flow vectors of consecutive frames is: ,in, Let k be the coordinates of the image light field particles at the initial moment, and k be the current pyramid level. This refers to the horizontal optical flow vector components of the image light field particles at the current pyramid level k, used to describe the amplitude and direction of motion in the horizontal direction. For the current pyramid level k, the calibrated image light field particles are the optical flow vector components in the vertical direction, used to describe the amplitude and direction of motion in the vertical direction.

7. An image recognition system for a camera according to claim 1, characterized in that: Before performing edge visual feature analysis on each target image, the texture recognition module also includes illumination normalization processing. The normalization processing includes: using the Retinex algorithm to separate the illumination of the target image into reflection and illumination components, suppressing global brightness fluctuations of the illumination component through logarithmic transformation, and then performing adaptive histogram equalization on the reflection component to enhance the local texture contrast of the reflection component; based on the processed reflection component, edge visual feature analysis is performed to generate multiple edge visual channels.

8. An image recognition sensor, characterized in that: The image recognition system applicable to a camera as described in claim 1, wherein the image recognition sensor is disposed in the image recognition module, and the image recognition sensor is used to realize the image information acquisition function of the image recognition module.

Images