Robot target intelligent identification method

Abstract

The invention belongs to the technical field of image recognition, and particularly relates to a robot target intelligent recognition method, which comprises the following steps: carrying out Gaussian mixture background modeling on matched pixel points between different frames of an investigation video to obtain foreground pixel points, background pixel points and a first Gaussian mixture model of the background pixel points; angular point matching is carried out on foreground areas among all the frames, Gaussian mixture model fitting is carried out on foreground pixel points according to a matching result, and a second Gaussian mixture model of all the foreground pixel points is obtained; according to the first Gaussian mixture model and the second Gaussian mixture model, the enhancement probability of the background pixel points and the enhancement probability of the foreground pixel points are determined respectively; determining the correction frequency of each gray value in each frame according to the enhancement probability, and performing histogram equalization on each frame to obtain an enhanced image; and performing target identification according to the enhanced image. According to the method, the contrast and detail expressive force of small targets in the video are effectively improved, and the accuracy of target detection and recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically, to a robot target intelligent recognition method. Background Art

[0002] With the rapid development of artificial intelligence and robotics, robot target recognition technology has become one of the core technologies in the fields of intelligent manufacturing, autonomous driving, smart security and military reconnaissance.

[0003] In robot reconnaissance missions, the reconnaissance video taken by the onboard camera has a problem of reduced clarity due to the robot's constant movement, which has a significant impact on the accurate identification of the target. Therefore, image enhancement processing of the reconnaissance video to improve video clarity is of great significance to improving the accuracy of target recognition.

[0004] Currently, the histogram equalization algorithm is widely used in the field of image enhancement due to its simple implementation and good enhancement effect. For example, Chinese patent documents with authorization announcement numbers CN114882346B and CN114092793B both use the histogram equalization algorithm to enhance images collected by robot cameras.

[0005] However, the histogram equalization algorithm has obvious limitations in the processing process: for grayscale features with high frequency, its enhancement effect is significant; but for grayscale features with low frequency, they may be over-compressed or even completely lost. In reconnaissance videos, since the targets to be identified are usually small, their corresponding grayscale features often have a low frequency of occurrence. If the histogram equalization is directly performed on each frame of the video, these key target information may be swallowed up, which will reduce the accuracy of target recognition. Summary of the invention

[0006] In order to solve the technical problem that the grayscale features with a relatively small frequency may be over-compressed or even completely lost, thereby reducing the accuracy of target recognition, the present invention provides a robot target intelligent recognition method, comprising: In the process of robot reconnaissance, the reconnaissance video is collected in real time through the camera carried by the robot; according to the moving speed and direction of the robot at each moment, the pixel points between different frames of the video are matched, and the mixed Gaussian background modeling is performed according to the matching results, and the pixel points in the video are divided into foreground pixel points and background pixel points to obtain the first mixed Gaussian model of the background pixel points; the foreground area composed of the foreground pixel points between each frame is matched with the corner points to obtain the corresponding relationship of the foreground pixel points between different frames, and the mixed Gaussian model is fitted according to the corresponding relationship of the foreground pixel points to obtain the second mixed Gaussian model of each foreground pixel point; the enhancement probability of the background pixel points and the foreground pixel points is determined according to the first mixed Gaussian model and the second mixed Gaussian model respectively; for any frame in the video, the correction frequency of each gray value in the frame is determined according to the enhancement probability of each pixel point in the frame, and the frame is histogram equalized according to the correction frequency of each gray value to obtain the enhanced image of the frame; the target is recognized according to the enhanced image of each frame.

[0007] The present invention can effectively establish a scene background model by real-time acquisition of reconnaissance video and using the robot's motion information to match pixels between frames. Based on the mixed Gaussian background modeling method, the video pixels are accurately divided into two categories: foreground and background, and a first mixed Gaussian model of background pixels is established, which not only achieves accurate extraction of the scene background, but also lays the foundation for subsequent target detection. The present invention performs corner point matching on the foreground area to obtain the correspondence between foreground pixels between different frames, and based on this, establishes a second mixed Gaussian model of foreground pixels, which helps to accurately capture the feature information of moving targets and improve the accuracy of foreground detection. The present invention calculates the enhancement probability of background pixels and foreground pixels respectively based on a first mixed Gaussian model of background pixels and a second mixed Gaussian model of foreground pixels, and corrects the frequency of each grayscale value based on the enhancement probability of each pixel, so that the frequency of stable grayscale features in multiple consecutive frames is increased in each frame, and avoids excessive compression or even complete loss of stable grayscale features in multiple consecutive frames due to too small frequency during enhancement, effectively improves the contrast and detail expression of small targets in the video, makes the target features more prominent, performs target recognition based on the enhanced image, improves the accuracy of target detection and recognition, and provides strong technical support for robot reconnaissance tasks.

[0008] Preferably, the pixel points between different frames of the video are matched according to the movement speed and direction of the robot at each moment, including: for any two adjacent frames in the video, according to the movement speed and direction of the robot at the corresponding moment and the time interval between the two frames, determining the pixel points corresponding to each pixel point in the previous frame in the next frame.

[0009] Preferably, the mixed Gaussian background modeling is performed according to the matching results, and the pixels in the video are divided into foreground pixels and background pixels to obtain a first mixed Gaussian model of the background pixels, including: the corresponding pixels in each frame constitute a pixel sequence, and the mixed Gaussian background modeling is performed on each pixel sequence to obtain a mixed Gaussian model of each pixel sequence, and the pixels in each pixel sequence are divided into foreground pixels and background pixels; the mixed Gaussian model of each pixel sequence is used as the first mixed Gaussian model corresponding to each background pixel in the pixel sequence.

[0010] The present invention performs mixed Gaussian background modeling on a pixel sequence and uses the mixed Gaussian model to reflect the grayscale statistical characteristics of the corresponding pixels in each frame. Based on the grayscale statistical characteristics, it can not only distinguish between foreground pixels and background pixels, but also provide a mathematical basis for the subsequent acquisition of the enhancement probability of background pixels.

[0011] Preferably, the method for acquiring the foreground area is: performing a connected domain analysis on the foreground pixels in each frame, and taking a connected domain containing foreground pixels whose number is greater than a preset number threshold as the foreground area.

[0012] The present invention determines the number of foreground pixel points contained in the connected domain, eliminates the interference of noise, and obtains a more accurate foreground area.

[0013] Preferably, the method of fitting a mixed Gaussian model according to the correspondence between foreground pixels to obtain a second mixed Gaussian model of each foreground pixel includes: forming a foreground pixel sequence from the corresponding foreground pixels in each frame; performing mixed Gaussian fitting on each foreground pixel sequence to obtain a mixed Gaussian model of each foreground pixel sequence, and using the mixed Gaussian model of each foreground pixel sequence as the second mixed Gaussian model corresponding to the foreground pixel in the foreground pixel sequence.

[0014] The present invention performs mixed Gaussian fitting on a foreground pixel sequence and uses a mixed Gaussian model to reflect the grayscale statistical characteristics of foreground pixels representing the same motion characteristics in each frame, thereby providing a mathematical basis for subsequently obtaining the enhancement probability of foreground pixels.

[0015] Preferably, the enhancement probability of the background pixel and the foreground pixel satisfies the expression: ;in, Indicates the target frame The enhancement probability of each pixel; Represents the set of foreground pixels in the target frame; Indicates the target frame pixels, when the When a pixel is a foreground pixel, , Indicates the target frame The weight of the sub-Gaussian model with the largest probability density under each sub-Gaussian model in the second mixed Gaussian model for each pixel point and the corresponding probability density; is the preset foreground parameter; when When pixels are background pixels, , Indicates the target frame The weight of the sub-Gaussian model with the largest probability density for each sub-Gaussian model in its first mixed Gaussian model and the corresponding probability density.

[0016] The present invention uses the weights and probability density of the sub-Gaussian model to characterize the grayscale statistical characteristics of pixel points in multiple consecutive frames, and assigns a larger enhancement probability to the stable grayscale features in multiple consecutive frames, so as to avoid the stable grayscale features in multiple consecutive frames being excessively compressed or even completely lost during enhancement due to their low frequency in a single frame, thereby effectively improving the reliability and stability of the enhancement results; the present invention takes into account the different feature requirements of the foreground and background areas, and can adjust the enhancement degree ratio of the foreground and background according to the specific application scenario, which can ensure the coordination of the enhancement effect, improve the overall quality and visual perception effect of the image, and provide a higher quality data foundation for subsequent target recognition.

[0017] Preferably, the method of determining the correction frequency of each grayscale value in the frame according to the enhancement probability of each pixel in the frame includes: determining the enhancement threshold of each grayscale value in the frame according to the enhancement probability of each pixel; in response to the frequency of the grayscale value being less than the enhancement threshold of the grayscale value, using the result of normalizing the enhancement threshold of the grayscale value as the correction frequency of the grayscale value; in response to the frequency of the grayscale value being not less than the enhancement threshold of the grayscale value, using the result of normalizing the frequency of the grayscale value as the correction frequency of the grayscale value.

[0018] The present invention corrects the frequency of the grayscale value according to the enhancement threshold of the grayscale value, so that the stable grayscale features in multiple consecutive frames can be enhanced emphatically to avoid excessive compression or loss of the grayscale features during enhancement due to their low frequency in a single frame, and ensures that the main grayscale features in each frame can be enhanced emphatically, making the target features more prominent, thereby providing a basis for subsequent target recognition.

[0019] Preferably, determining the enhancement threshold of each grayscale value in the frame according to the enhancement probability of each pixel point includes: for any grayscale value in any frame, taking the average of the enhancement probabilities of all pixels corresponding to the grayscale value in the frame as the enhancement threshold of the grayscale value.

[0020] Preferably, the corner point matching adopts a SIFT feature matching algorithm.

[0021] Preferably, the target recognition based on the enhanced image of each frame includes: inputting the enhanced image of each frame into a trained neural network and outputting the recognition result.

[0022] The beneficial effects of the present invention are as follows: the present invention realizes accurate extraction of scene background, laying a foundation for subsequent target detection. The present invention accurately captures the characteristic information of moving targets and improves the accuracy of foreground detection. The present invention avoids excessive compression or even complete loss of enhancement due to the low frequency of stable grayscale features in multiple consecutive frames, effectively improves the contrast and detail expression of small targets in the video, makes the target features more prominent, improves the accuracy of target detection and recognition, and provides strong technical support for robot reconnaissance tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart schematically illustrating a method for intelligently identifying a robot target in the present invention; Figure 2 Schematically shows a frame of image in the reconnaissance video of the present invention; Figure 3 is a flow chart schematically illustrating step S2 of a robot target intelligent recognition method in the present invention; Figure 4 is a schematic diagram schematically showing a foreground area in the present invention; Figure 5 is a flow chart schematically illustrating step S3 of a robot target intelligent recognition method in the present invention; Figure 6 is a schematic diagram showing the correction frequency of each gray value. Figure 2 The enhanced image obtained after histogram equalization; Figure 7 is a schematic diagram showing the frequency pair before correction according to each gray value Figure 2 The enhanced image obtained after histogram equalization. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0026] The embodiment of the present invention discloses a robot target intelligent recognition method, referring to Figure 1 , including steps S1 to S4: S1. During the robot reconnaissance process, the camera mounted on the robot collects reconnaissance videos in real time.

[0027] In the present invention, the acquisition frequency is 0.02 frames per second, and the implementer can set the acquisition frequency according to the actual implementation situation. For the convenience of processing, each frame of the acquired video is a grayscale image. Figure 2 A frame of image in the detection video.

[0028] S2. Match the pixels between different frames of the video, perform mixed Gaussian background modeling based on the matching results, divide the pixels in the video into background pixels and foreground pixels, and obtain the first mixed Gaussian model of the background pixels and the second mixed Gaussian model of the foreground pixels respectively.

[0029] See the flowchart of step S2. Figure 3 , including steps S201 to S204, specifically: S201, matching pixel points between different frames of the video according to the moving speed and direction of the robot at each moment.

[0030] It should be noted that since the robot is constantly moving, the objects in the video are moving relative to the lens of the camera mounted on the robot. Therefore, the present invention matches the pixel points between two adjacent frames according to the time interval between the two adjacent frames and the movement speed and direction of the robot at the corresponding moment.

[0031] Specifically, for any two adjacent frames in the video, the motion model of the camera on the robot during this period is constructed based on the time interval between the two adjacent frames, the movement speed and direction of the robot at the corresponding moment. According to the intrinsic parameters of the camera, the pixel coordinates of the previous frame are converted into three-dimensional world coordinates. According to the camera's motion model, the three-dimensional world coordinates are reprojected to the image plane of the next frame using the camera's intrinsic parameters, and the corresponding pixel position in the next frame can be obtained. In this way, the corresponding pixel points of each pixel point in the previous frame in the next frame can be obtained.

[0032] It should be noted that constructing a camera motion model, converting pixel coordinates into three-dimensional world coordinates, and projecting the three-dimensional world coordinates onto an image plane are well-known technologies and will not be described in detail herein.

[0033] It should be further explained that, when the robot moves forward, the distance between the object in front of the robot and the robot shortens. Based on the principle that objects are smaller when farther away and larger when closer, the object in the latter frame of the video becomes larger than the object in the previous frame. In this case, one pixel in the previous frame may correspond to multiple pixels in the latter frame. When the robot moves backward, the distance between the object in front of the robot and the robot increases. The object in the latter frame of the video becomes smaller than the object in the previous frame. In this case, multiple pixels in the previous frame may correspond to one pixel in the latter frame.

[0034] S202, performing mixed Gaussian background modeling according to the matching results, dividing the pixels in the video into foreground pixels and background pixels, and obtaining a first mixed Gaussian model of the background pixels.

[0035] It should be noted that the mixed Gaussian background modeling constructs a mixed Gaussian model for the pixel points at the same position in each frame of the video. Since the robot is mobile, the camera mounted on the robot is also mobile, resulting in the pixel points at the same position in each frame of the video representing different image features. Therefore, the present invention constructs a pixel point sequence according to the correspondence between the pixel points in each frame of the video obtained in step S201, and performs Gaussian background modeling on the pixel point sequence.

[0036] Specifically, the corresponding pixels in each frame constitute a pixel sequence. For example, pixel A1 in the first frame corresponds to pixel A2 and B2 in the second frame, pixel A2 in the second frame corresponds to pixel A3 and C3 in the third frame, and pixel B2 in the second frame corresponds to pixel B3 and D3 in the third frame. Then A1, A2, B2, A3, C3, B3, and D3 constitute a pixel sequence.

[0037] A mixed Gaussian background model is performed on each pixel sequence to obtain a mixed Gaussian model of each pixel sequence, and the pixels in each pixel sequence are divided into foreground pixels and background pixels. The mixed Gaussian model of each pixel sequence is the first mixed Gaussian model corresponding to each background pixel in the pixel sequence.

[0038] It should be noted that the number of sub-Gaussian models in the mixed Gaussian background modeling in the present invention is set to 5, and the implementer can set the number of sub-Gaussian models according to the actual implementation situation. Distinguishing foreground pixels and background pixels is a well-known technology in the mixed Gaussian modeling technology, which will not be repeated here.

[0039] S203 , performing corner point matching on the foreground area formed by the foreground pixels between the frames to obtain the corresponding relationship between the foreground pixels between different frames.

[0040] It should be noted that the foreground pixel points are the pixel points corresponding to the moving object. In different frames, the position of the moving object relative to the camera lens of the robot is different, so that the position and size of the area where the moving object is located in different frames are different. Therefore, the present invention matches the foreground pixel points between different frames.

[0041] Specifically, a connected domain analysis is performed on the foreground pixels in each frame, and the connected domain containing foreground pixels whose number is greater than a preset threshold is taken as the foreground area, and each pixel in the connected domain containing foreground pixel data not greater than the preset threshold is taken as a noise point, and the noise points are reclassified as background pixels. Figure 4 Schematic diagram of the foreground area.

[0042] In the present invention, the quantity threshold is set by implementers according to actual implementation conditions, for example, 2.

[0043] Corner point matching is performed on the foreground area between two adjacent frames to obtain matching feature points in the foreground area of ​​the two adjacent frames. The geometric transformation parameters between the foreground areas of the two adjacent frames are calculated using the matching feature point pairs. Based on the geometric transformation parameters, the correspondence between the foreground pixels of non-feature points between the two adjacent frames is derived through interpolation operations.

[0044] It should be noted that the present invention uses the Scale Invariant Feature Transform (SIFT) algorithm to match the corners of the foreground area between two adjacent frames, uses the Random Sampling Consensus (RANSAC) algorithm to calculate the geometric transformation parameters between the foreground areas of two adjacent frames, and derives the correspondence between the foreground pixels of non-feature points between two adjacent frames based on bilinear interpolation. The implementer can select the corner matching algorithm, the algorithm for solving the geometric transformation parameters, and the algorithm for interpolation operation according to the actual implementation situation. The SIFT algorithm, the RANSAC algorithm, and the bilinear interpolation are well-known technologies and will not be described in detail here.

[0045] It should be further explained that, in the previous and next frames, the position of the object corresponding to the foreground pixel changes relative to the position of the robot. When the distance from the object corresponding to the foreground pixel to the robot increases, based on the principle that objects are smaller when farther away and larger when closer, the object corresponding to the foreground pixel becomes smaller in the next frame, and multiple foreground pixels in the previous frame may correspond to one foreground pixel in the next frame; when the distance from the object corresponding to the foreground pixel to the robot decreases, the object corresponding to the foreground pixel becomes larger in the next frame, and one foreground pixel in the previous frame may correspond to multiple foreground pixels in the next frame.

[0046] S204 , performing mixed Gaussian model fitting according to the correspondence between foreground pixels of each frame to obtain a second mixed Gaussian model of each foreground pixel.

[0047] The corresponding foreground pixels in each frame constitute a foreground pixel sequence. Mixed Gaussian fitting is performed on each foreground pixel sequence to obtain a mixed Gaussian model of each foreground pixel sequence, and the mixed Gaussian model of each foreground pixel sequence is the second mixed Gaussian model corresponding to the foreground pixel in the foreground pixel sequence.

[0048] It should be noted that the present invention performs mixed Gaussian fitting on each foreground pixel sequence respectively, and adopts the least square method. The implementer can select the fitting algorithm according to the actual implementation situation, such as the maximum likelihood method. In the present invention, when mixed Gaussian fitting is performed on each foreground pixel sequence respectively, the number of sub-Gaussian models included in the mixed Gaussian model is 5, and the implementer can adjust the number of sub-Gaussian models according to the actual implementation situation.

[0049] S3. Enhance each frame according to the Gaussian mixture model corresponding to the pixels in each frame of the video and the pixel type.

[0050] See the flowchart of step S3. Figure 5 , including steps S301 to S303, specifically: S301 , taking any frame as a target frame, and determining enhancement probabilities of background pixels and foreground pixels in the target frame according to a first mixed Gaussian model and a second mixed Gaussian model respectively.

[0051] Specifically, any frame is taken as the target frame, and the enhancement probability of each pixel in the target frame satisfies the expression: ; in, Indicates the target frame The enhancement probability of each pixel; Represents the set of foreground pixels in the target frame; Indicates the target frame pixels, when the When a pixel is a foreground pixel, Indicates the target frame The weight of the sub-Gaussian model with the largest probability density for each sub-Gaussian model in its second mixed Gaussian model, Indicates the target frame The probability density of the pixel point corresponding to the sub-Gaussian model with the largest probability density under each sub-Gaussian model in its second mixed Gaussian model; is the preset foreground parameter, which is used to adjust the importance of the foreground; When pixels are background pixels, Indicates the target frame The weight of the sub-Gaussian model with the largest probability density for each sub-Gaussian model in the first mixed Gaussian model, Indicates the target frame The corresponding probability density of the sub-Gaussian model with the largest probability density under each sub-Gaussian model in its first mixed Gaussian model is the pixel point.

[0052] When the target frame pixels are foreground pixels, and the first The greater the probability density of a pixel point in a sub-Gaussian model of its second mixed Gaussian model, the greater the probability density of the The more the pixel point conforms to the foreground feature corresponding to the sub-Gaussian model, the greater the weight of the sub-Gaussian model, which means that in multiple consecutive frames, the moving object has shown the foreground feature. The foreground feature is a stable grayscale feature in multiple consecutive frames. The foreground feature is an important feature of the moving object. In order to make the moving object clearer, the pixel points corresponding to the foreground feature need to be enhanced. And weight Get the The enhancement probability of pixels is And the weight of the corresponding sub-Gaussian model The larger the The greater the enhancement probability of each pixel.

[0053] When the target frame pixels are background pixels, and the first The greater the probability density of a pixel point in a sub-Gaussian model of its first mixed Gaussian model, the greater the probability density of the The more the pixel point conforms to the background feature corresponding to the sub-Gaussian model, the greater the weight of the sub-Gaussian model, which means that the background object has shown the background feature in multiple consecutive frames. The background feature is a stable grayscale feature in multiple consecutive frames. The background feature is an important feature of the object in the background. In order to make the background clearer, the pixel point corresponding to the background feature needs to be enhanced. And weight Get the The enhancement probability of pixels is And the weight of the corresponding sub-Gaussian model The larger the The greater the enhancement probability of each pixel.

[0054] In the formula, the foreground parameter Used to adjust the importance of the foreground. In the present invention, the foreground object in motion is the main target of the robot's reconnaissance. The foreground pixels of the target frame need to be enhanced. Therefore, in the present invention, In other scenarios, the target to be identified may be static, so the implementer can set it according to the actual implementation situation. .

[0055] It should be noted that, as the robot moves, some areas may disappear within the robot's field of view, and some areas may appear within the robot's field of view, so that some pixels in the previous frame have no corresponding pixels in the next frame, and some pixels in the next frame have no corresponding pixels in the previous frame, so the lengths of the pixel sequences obtained in step S202 are different. Similarly, as the foreground corresponding object moves, some foreground areas may disappear within the robot's field of view, and some foreground areas may appear within the robot's field of view, so that some foreground pixels in the previous frame have no corresponding foreground pixels in the next frame, and some foreground pixels in the next frame have no corresponding foreground pixels in the previous frame, so the lengths of the foreground pixel sequences obtained in step S204 are different. When the length of the pixel sequence is too small, the accuracy of mixed Gaussian background modeling of the pixel sequence is poor, and when the length of the foreground pixel sequence is too small, the accuracy of mixed Gaussian fitting of the foreground pixel sequence is poor. Therefore, in another embodiment, when the length of a pixel point sequence or a foreground pixel point sequence is too small, mixed Gaussian background modeling or mixed Gaussian fitting is not performed on the pixel point sequence or the foreground pixel point sequence, and the enhancement probability of each pixel point or foreground pixel point in the pixel point sequence or the foreground pixel point sequence is directly set.

[0056] In another embodiment, in response to the length of the pixel sequence being not greater than a preset length threshold, the mixed Gaussian background modeling is not performed on the pixel sequence, and for each pixel in the pixel sequence, the enhancement probability of each pixel is specified as In response to the length of the foreground pixel sequence being not greater than the preset length threshold, the mixed Gaussian fitting is not performed on the foreground pixel sequence, and the enhancement probability of each foreground pixel in the foreground pixel sequence is specified as In this embodiment, the length threshold is set by the implementer according to the actual implementation situation, for example, the length threshold is 5. is the preset foreground parameter, and the empirical value is , implementers can set according to the actual implementation situation .

[0057] S302 : Determine an enhancement threshold of each gray value in the target frame according to the enhancement probability of each pixel in the target frame.

[0058] It should be noted that when the enhancement probability of each pixel corresponding to a grayscale value in the target frame is greater, it means that the grayscale value is a stable grayscale feature in multiple consecutive frames, and the grayscale value needs to be enhanced. Therefore, the present invention sets the enhancement threshold of each grayscale value according to the enhancement probability of each pixel corresponding to each grayscale value, so that when the frequency of each grayscale value is subsequently corrected using the enhancement threshold of each grayscale value, the grayscale value with a larger enhancement threshold has a greater correction frequency, thereby achieving focused enhancement of the grayscale value.

[0059] Specifically, the enhancement threshold of each gray value satisfies the expression: ; in, Represents the grayscale value in the target frame The enhancement threshold; Indicates that the gray value in the target frame is The enhancement probability of the s-th pixel; Indicates that the gray value in the target frame is When the gray value in the target frame is The greater the enhancement probability of each pixel point, the gray value in the target frame The larger the enhancement threshold, the gray value will be adjusted according to the enhancement threshold. The larger the correction frequency obtained after the frequency correction, the more gray values ​​can be enhanced when the histogram equalization is performed on the target frame according to the correction frequency of each gray value. The corresponding image features.

[0060] S303 , determining the correction frequency of each gray value according to the enhancement threshold of each gray value, performing histogram equalization on the target frame according to the correction frequency of each gray value, and obtaining an enhanced image of the target frame.

[0061] In response to the grayscale value frequency being less than the grayscale value enhancement threshold, the grayscale value enhancement threshold normalized result is used as the grayscale value correction frequency; in response to the grayscale value frequency being not less than the grayscale value enhancement threshold, the grayscale value frequency normalized result is used as the grayscale value correction frequency.

[0062] Specifically, the correction frequency of each gray value satisfies the expression: ; in, Represents the grayscale value in the target frame The frequency of correction; Represents the grayscale value in the target frame The enhancement threshold; Represents the grayscale value in the target frame frequency; Represents the grayscale value in the target frame The frequency process quantity, when the gray value The frequency is not less than the gray value When the enhancement threshold is , the frequency process quantity is taken as the gray value The frequency of gray value The frequency is less than the gray value When the enhancement threshold is , the frequency process quantity is taken as the gray value The enhancement threshold; when the gray value The larger the frequency, the gray value is the main grayscale feature in the target frame. The larger the enhancement threshold, the gray value is a stable grayscale feature in multiple consecutive frames; therefore, in the present invention, when the grayscale value When the frequency is less than the enhancement threshold, the gray value The correction frequency is positively correlated with the enhancement threshold. When the frequency is not less than the enhancement threshold, the gray value Correction frequency and gray value The original frequencies are positively correlated, which ensures that during subsequent enhancement, the main grayscale features in the target frame and the stable grayscale features in multiple consecutive frames can be enhanced.

[0063] With the grayscale value as the horizontal axis and the correction frequency of the grayscale value as the vertical axis, a grayscale correction histogram is constructed, and histogram equalization is performed on the grayscale correction histogram to achieve enhancement of the target frame.

[0064] Similarly, the enhanced image of each frame in the video is obtained. Figure 6 Based on Figure 2 The correction frequency of each gray value in Figure 2 The enhanced image obtained after histogram equalization, Figure 7 Based on Figure 2 The frequency of each gray value before correction is Figure 2 The enhanced image is obtained after histogram equalization. It can be seen that Figure 7 The background is over-enhanced, the foreground is blurred and difficult to identify, and Figure 6 The mid-foreground enhancement effect is good and easier to identify.

[0065] S4. Perform target recognition based on the enhanced images of each frame.

[0066] Specifically, the present invention uses the YOLOv5 network to identify targets in enhanced images, including drones, vehicles, etc. Implementers can select neural networks and set targets to be identified according to actual conditions.

Claims

1. A robot target intelligent recognition method, characterized in that: include: During the robot reconnaissance process, the camera on the robot collects reconnaissance videos in real time; According to the robot's moving speed and direction at each moment, the pixels between different frames of the video are matched, and a mixed Gaussian background model is performed based on the matching results. The pixels in the video are divided into foreground pixels and background pixels to obtain the first mixed Gaussian model of the background pixels. Corner point matching is performed on the foreground area composed of foreground pixels between frames to obtain the correspondence between foreground pixels between different frames, and a mixed Gaussian model is fitted according to the correspondence between foreground pixels to obtain a second mixed Gaussian model of each foreground pixel; Determine the enhancement probability of background pixels and foreground pixels respectively according to the first mixed Gaussian model and the second mixed Gaussian model; For any frame in the video, the correction frequency of each grayscale value in the frame is determined according to the enhancement probability of each pixel in the frame, and the frame is histogram equalized according to the correction frequency of each grayscale value to obtain an enhanced image of the frame; target recognition is performed based on the enhanced images of each frame.

2. A robot target intelligent recognition method according to claim 1, characterized in that: The matching of pixels between different frames of the video according to the moving speed and direction of the robot at each moment includes: For any two adjacent frames in the video, the corresponding pixel points in the next frame are determined according to the robot's moving speed and direction at the corresponding moment and the time interval between the two frames.

3. A robot target intelligent recognition method according to claim 1 or 2, characterized in that: The mixed Gaussian background modeling is performed according to the matching result, and the pixels in the video are divided into foreground pixels and background pixels to obtain a first mixed Gaussian model of the background pixels, including: The corresponding pixels in each frame constitute a pixel sequence, and a mixed Gaussian background model is performed on each pixel sequence to obtain a mixed Gaussian model of each pixel sequence, and the pixels in each pixel sequence are divided into foreground pixels and background pixels; the mixed Gaussian model of each pixel sequence is used as the first mixed Gaussian model corresponding to each background pixel in the pixel sequence.

4. A robot target intelligent recognition method according to claim 1, characterized in that: The method for obtaining the foreground area is: A connected domain analysis is performed on the foreground pixels in each frame, and a connected domain containing foreground pixels whose number is greater than a preset threshold is taken as a foreground area.

5. A robot target intelligent recognition method according to claim 1, characterized in that: The method of fitting a mixed Gaussian model according to the corresponding relationship of the foreground pixels to obtain a second mixed Gaussian model of each foreground pixel includes: The corresponding foreground pixel points in each frame constitute a foreground pixel point sequence; mixed Gaussian fitting is performed on each foreground pixel point sequence to obtain a mixed Gaussian model of each foreground pixel point sequence, and the mixed Gaussian model of each foreground pixel point sequence is used as the second mixed Gaussian model corresponding to the foreground pixel point in the foreground pixel point sequence.

6. A robot target intelligent recognition method according to claim 1, characterized in that: The enhancement probability of the background pixel and the foreground pixel satisfies the expression: ； in, Indicates the target frame The enhancement probability of each pixel; Represents the set of foreground pixels in the target frame; Indicates the target frame pixels, when the When a pixel is a foreground pixel, , Indicates the target frame The weight of the sub-Gaussian model with the largest probability density under each sub-Gaussian model in the second mixed Gaussian model for each pixel point and the corresponding probability density; is the preset foreground parameter; when When pixels are background pixels, , Indicates the target frame The weight of the sub-Gaussian model with the largest probability density for each sub-Gaussian model in its first mixed Gaussian model and the corresponding probability density.

7. A robot target intelligent recognition method according to claim 1 or 6, characterized in that: Determining the correction frequency of each gray value in the frame according to the enhancement probability of each pixel in the frame includes: Determine the enhancement threshold of each gray value in the frame according to the enhancement probability of each pixel; In response to the grayscale value frequency being less than the grayscale value enhancement threshold, the grayscale value enhancement threshold normalized result is used as the grayscale value correction frequency; in response to the grayscale value frequency being not less than the grayscale value enhancement threshold, the grayscale value frequency normalized result is used as the grayscale value correction frequency.

8. A robot target intelligent recognition method according to claim 7, characterized in that: Determining the enhancement threshold of each gray value in the frame according to the enhancement probability of each pixel point includes: For any gray value in any frame, the average value of the enhancement probabilities of all pixels corresponding to the gray value in the frame is used as the enhancement threshold of the gray value.

9. A robot target intelligent recognition method according to claim 1, characterized in that: The corner point matching adopts the SIFT feature matching algorithm.

10. The method for intelligently identifying a robot target according to claim 1, characterized in that: The target recognition is performed according to the enhanced image of each frame, including: The enhanced images of each frame are input into the trained neural network and the recognition results are output.

Images