Target recognition method, device, electronic device and storage medium

Abstract

The present invention provides a target recognition method, device, electronic device, and storage medium, wherein the method includes: determining a monitoring image sequence, performing target recognition on each frame image in the monitoring image sequence, and obtaining the target position of the detection target in each frame image; determining a trajectory set of the monitoring image sequence based on the target position of each detection target in multiple consecutive frames of the monitoring image sequence; and comparing the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result. The method, device, electronic device, and storage medium provided by the present invention can generate a trajectory set for each detection target in a real-time monitoring image sequence, and then compare it with an initially generated template trajectory set to obtain a target recognition result, thereby achieving dynamic comparative recognition of the detection target trajectory of the monitored object and the template trajectory, improving the accuracy of detection of each detection target, and thereby improving the recognition effect of the detection target.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a target recognition method, device, electronic device and storage medium. Background Art

[0002] In the knitting industry, the production process of grey fabric is divided into weft knitting and warp knitting. In the weft knitting production process, defects on the fabric surface due to damage to the knitting needles and other reasons are considered inferior fabrics.

[0003] Currently, there are two main methods for detecting defective weft knitted fabrics: Method 1) Using infrared lasers to determine whether the knitting needles are damaged based on whether the light is reflected, thereby determining the production status; Method 2) Using imaging equipment, traditional algorithms are used to segment the target in the captured image. The rotation speed and initial fabric surface information are recorded during the registration stage, and the segmented target is compared with the initial fabric surface information and converted to obtain the defect target location.

[0004] However, the infrared laser method in method 1) has poor stability. Due to the large amount of floating cotton in the weft knitting production environment, it is easy to block the receiver and cause misidentification, resulting in frequent downtime and affecting production. In method 2), the traditional algorithm for target segmentation does not establish the spatiotemporal relationship of the targets in each frame, making it difficult to determine whether the targets in each frame are the same, which in turn leads to significant limitations in target recognition. Summary of the Invention

[0005] The present invention provides a target recognition method, device, electronic device and storage medium, which are used to solve the defects of traditional algorithms in target recognition in the prior art, such as poor applicability, ease of use and poor recognition effect.

[0006] The present invention provides a target recognition method, comprising:

[0007] Determine a monitoring image sequence, perform target recognition on each frame image in the monitoring image sequence, and obtain a target position of a detection target in each frame image;

[0008] determining a trajectory set of the monitoring image sequence based on a target position of each detection target in a plurality of consecutive frames of images in the monitoring image sequence;

[0009] The template trajectory set is compared with the trajectory set of the monitoring image sequence to obtain a target recognition result, wherein the template trajectory set is determined based on the target position of each initial target in the continuous multiple frames of the initial image sequence.

[0010] According to a target recognition method provided by the present invention, determining a trajectory set of the monitoring image sequence based on the target position of each detection target in a plurality of consecutive frames of images in the monitoring image sequence includes:

[0011] performing trajectory matching on the detected target in the current frame image based on the target position of the detected target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image;

[0012] The successfully matched detection targets are added to the corresponding tracks, a new track is constructed based on the detection targets that failed to match, and the next frame image of the current frame image is updated as the current frame image.

[0013] According to a target recognition method provided by the present invention, based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, trajectory matching of the detection target in the current frame image is performed, including:

[0014] Based on the target position of the detection target in the current frame image of the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of each trajectory in the trajectory set, the detection target in the current frame image is tracked and matched; the extension direction is determined based on the sequence and positional relationship of each target in the corresponding trajectory.

[0015] According to a target recognition method provided by the present invention, based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of the trajectory, the trajectory matching of the detection target in the current frame image is performed, including:

[0016] Determining a position matching result of the detected target in the current frame image based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory;

[0017] determining, based on a center position of a target position of a detection target in the current frame image, an estimated extension direction of the detection target in the current frame image and an end point in each trajectory, and determining a trajectory consistency state based on the extension direction of each trajectory and the estimated extension direction;

[0018] Determining a temporal continuity state based on the number of frames between the current frame image and the frame image where the tail point target in each trajectory is located;

[0019] Based on the position matching result, the trajectory consistency state and the time continuity state, trajectory matching is performed on the detection target in the current frame image.

[0020] According to a target recognition method provided by the present invention, determining a position matching result of a detected target in the current frame image based on width and height information in a target position of the detected target in the current frame image and width and height information in target position information of a tail point target in each trajectory includes:

[0021] Determining a height intersection-and-joint ratio and a width intersection-and-joint ratio of the detected target in the current frame image and the tail point target in each trajectory based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory;

[0022] Determining a matching loss value between the detected target in the current frame image and the tail point target in each trajectory based on an intersection-over-union ratio of heights and an intersection-over-union ratio of widths;

[0023] The position matching result is determined based on the matching loss value of the detected target in the current frame image and the tail point target in each trajectory.

[0024] According to a target recognition method provided by the present invention, the template trajectory set is compared with the trajectory set of the monitoring image sequence to obtain a target recognition result, including:

[0025] Calculate the intersection-and-union ratio of any trajectory in the trajectory set of the monitoring image sequence and each trajectory in the template trajectory set, and obtain a matching loss value between the any trajectory and each trajectory in the template trajectory set;

[0026] Based on the matching loss value of each track in the track set of the monitoring image sequence and each track in the template track set, the number of unsuccessfully matched tracks is determined as the target recognition result.

[0027] According to a target recognition method provided by the present invention, the template trajectory set is determined based on the following steps:

[0028] determining a plurality of groups of initial image sequences;

[0029] determining a trajectory set of any initial image sequence based on target positions of each detection target in a plurality of consecutive frames of images in any initial image sequence in the plurality of groups of initial image sequences;

[0030] calculating an intersection-over-union ratio for each track in the track set of each initial image sequence in the plurality of groups of initial image sequences, and obtaining a matching value between each track;

[0031] The template trajectory set is determined based on the matching values ​​between each trajectory and a preset threshold.

[0032] The present invention also provides a target recognition device, comprising:

[0033] a determination module, configured to determine a monitoring image sequence, perform target recognition on each frame image in the monitoring image sequence, and obtain a target position of a detection target in each frame image;

[0034] a trajectory generation module, configured to determine a trajectory set of the monitoring image sequence based on a target position of each detection target in a plurality of consecutive frames of the monitoring image sequence;

[0035] The recognition module is used to compare a template trajectory set with a trajectory set of the monitoring image sequence to obtain a target recognition result, wherein the template trajectory set is determined based on the target position of each initial target in a plurality of consecutive frames of images in the initial image sequence.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the target recognition methods described above are implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the target recognition methods described above when executed by a processor.

[0038] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the target recognition methods described above are implemented.

[0039] The present invention provides a target recognition method, device, electronic device and storage medium. By generating a trajectory set for each detection target in a real-time monitoring image sequence and then comparing it with an initially generated template trajectory set, a target recognition result is obtained. Dynamic comparative recognition of the detection target trajectory and template trajectory of the monitored object is achieved, thereby improving the accuracy of determining whether each detection target is actually the same detection target, and thus improving the reliability and accuracy of identifying multiple detection targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 It is a flowchart of the target recognition method provided by the present invention;

[0042] Figure 2 1 is a flow chart of the trajectory set generation method provided by the present invention;

[0043] Figure 3 1 is a flow chart of the trajectory matching method provided by the present invention;

[0044] Figure 4 is a flow chart of the position matching method provided by the present invention;

[0045] Figure 5 Schematic diagram of the process of obtaining target recognition results provided by the present invention;

[0046] Figure 6 It is a flow chart of the method for obtaining a template set provided by the present invention;

[0047] Figure 7 is a top view schematic diagram of the rotation of the object to be monitored provided by the present invention;

[0048] Figure 8 This is a schematic diagram of the template trajectory set generation stage provided by the present invention;

[0049] Figure 9 Schematic diagram of the structure of the target detector provided by the present invention;

[0050] Figure 10 It is a schematic diagram of the processing flow of the inter-frame matcher provided by the present invention;

[0051] Figure 11 It is a schematic diagram of the target recognition stage in the monitoring image sequence provided by the present invention;

[0052] Figure 12 A schematic structural diagram of a target recognition device provided by the present invention;

[0053] Figure 13 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0055] At present, the traditional algorithm for segmenting targets requires the pre-evaluation of the loom speed and the storage of the initial state information of the cloth. Once the speed changes, recalibration is required. Otherwise, after the loom speed changes, when comparing the detection targets in the two frames, the loom speed before the change will be used for conversion. As a result, the same detection target in the two frames is judged as two detection targets, or different detection targets in the two frames are judged as the same detection target, resulting in erroneous recognition results. At the same time, the traditional algorithm does not construct the spatiotemporal relationship between the detection targets in multiple frames. As a result, it can only identify whether the detection targets are the same detection target based on the pre-evaluated loom speed, making it difficult to determine whether the detection targets in each frame are the same. For example, if there is a detection target in each of the two frames and they are not actually the same detection target, but the positions of the two detection targets in the two frames are close or the same, the traditional algorithm will judge them as the same detection target, resulting in recognition errors. It can be seen that the recognition effect of the detection targets of the traditional algorithm is very limited.

[0056] Therefore, how to improve the recognition effect of detection targets is a technical problem that needs to be solved urgently in this field.

[0057] In view of the above situation, an embodiment of the present invention provides a target recognition method. Figure 1 FIG. 1 is a flow chart of the target recognition method provided by the present invention. Figure 1 As shown, the method includes:

[0058] Step 110: determining a monitoring image sequence, performing target recognition on each frame image in the monitoring image sequence, and obtaining a target position of a detection target in each frame image;

[0059] Specifically, in step 110 , target recognition is performed on each frame of the monitoring image sequence according to the input monitoring image sequence to obtain the target position of the detection target in each frame of the image.

[0060] It should be noted that the monitoring image sequence is a sequence of images of continuous frames in the same direction and the same size, wherein each frame image may contain the entire monitored object at different angles. The monitored object here may be the cloth surface on the loom, or other periodically rotating objects. A set of monitoring image sequences is generated for each rotation of the monitored object. When monitoring the cloth surface of the loom, the detection target obtained by target recognition may be a white area caused by a broken needle, a dark area caused by a disordered needle, or a starting mark area, etc. The target position of the detection target in each frame image can be shown as the coordinate value of the center position of the detection frame of the detection target in the image and the length and width of the detection frame in the image, wherein the detection frame can be the circumscribed rectangle of the detection target.

[0061] Step 120 , determining a trajectory set of the monitoring image sequence based on the target position of each detection target in a plurality of consecutive frames of the monitoring image sequence;

[0062] Step 130 : Compare the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result. The template trajectory set is determined based on the target position of each initial target in the continuous multiple frames of the initial image sequence.

[0063] In two different frames of images in a monitoring image sequence, there may be two actually different detection targets, which appear at the same coordinate position or close coordinate positions in the two frames of images, and the above situation is very likely to lead to misjudgment. Considering that in multiple frames of images taken continuously in an image sequence, if there is a detection target being monitored, then the detection target exists in the continuous multiple frames of images, and the positions of the detection targets in the continuous multiple frames of images are relatively close. It can be seen that the misjudgment in the above situation must be caused by the large difference in shooting time between the two selected frames of images. It can be seen that when the monitored object rotates, the different detection targets that are actually adjacent to the detected object must have different trajectories in the monitoring image sequence generated by one rotation of the monitored object. Therefore, the embodiment of the present invention performs trajectory detection based on each detection target in the initial image sequence and the monitoring image sequence.

[0064] Specifically, after starting the monitoring operation of the monitored object, the target position of each detection target in the initial image sequence is identified through the initial image sequence captured by the shooting device, and the trajectory matching is performed based on the target position of each detection target in the continuous frame images of the initial image sequence to obtain a template trajectory set; after obtaining the template trajectory set, the target position of each detection target in the monitoring image sequence captured by the shooting device is identified, and the trajectory matching is performed based on the target position of each detection target in the continuous frame images of the monitoring image sequence to obtain a trajectory set of the monitoring image sequence; finally, the trajectory set of the monitoring image sequence and the template trajectory set are compared to obtain the target recognition result.

[0065] It should be noted that the target positions of each detection target in the continuous frames of the initial image sequence and the target positions of each detection target in the continuous frames of the monitoring image sequence are all track detection. The target position of each detection target in each frame of the image sequence will only be track matched with the target position of the detection target in each frame of the continuous frames adjacent to the frame (within a preset number of frames). If the frame is preceded and followed by other images in the image sequence by more than a preset number of frames, it is considered that the detection targets in the frame are not actually the same detection targets. The trajectory matching can be track matching of each target position in each frame of the continuous frames adjacent to the frame image in which the current detection target is located, or it can be track matching based on the target position of the end point target of each trajectory in the trajectory set. The embodiment of the present invention does not limit this. The initial image sequence can be a group of image sequences or multiple groups of image sequences. If the initial image sequence contains multiple groups of image sequences, the template trajectory set can be obtained by merging the trajectory sets generated by each group of image sequences. The embodiment of the present invention does not limit this.

[0066] The comparison between the trajectory set of the monitoring image sequence and the template trajectory set can be performed by comparing the number of trajectories in the monitoring image sequence and the template trajectory set, with the target recognition result being whether the number of trajectories is the same. Alternatively, each trajectory in the monitoring image sequence trajectory set and each trajectory in the template trajectory set are matched for target similarity, with the number of detected targets for which the trajectories do not successfully match being the target recognition result. This is not limited in the present embodiment. The target recognition result is a determination of whether the monitored object has generated a new detection target, that is, whether an abnormal detection target has been generated.

[0067] The target recognition method provided by the embodiment of the present invention obtains a target recognition result by generating a trajectory set for each detection target in a real-time monitoring image sequence, and then comparing it with the initially generated template trajectory set. This realizes the dynamic comparative recognition of the detection target trajectory and template trajectory of the monitored object, improves the accuracy of judging whether each detection target is actually the same detection target, and thereby improves the reliability and accuracy of identifying multiple detection targets.

[0068] In particular, when the above-mentioned target recognition method is applied to defective cloth detection, abnormal areas on the cloth surface can be used as detection targets. By monitoring the changes in the number of unmatched trajectories obtained by comparing and identifying the trajectory set of the image sequence and the template trajectory set, it is determined whether there are new abnormal areas, thereby interrupting the abnormal operation of the loom in time.

[0069] Based on the above embodiments, Figure 2 FIG. 1 is a flow chart of the trajectory set generation method provided by the present invention. Figure 2 As shown, step 120 includes:

[0070] Step 121 , performing trajectory matching on the detected target in the current frame image based on the target position of the detected target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of frames between the frame image containing the tail point target in each trajectory and the current frame image;

[0071] In step 122 , the successfully matched detection targets are added to the corresponding tracks, a new track is constructed based on the unmatched detection targets, and the next frame image of the current frame image is updated as the current frame image.

[0072] Considering that if the trajectory of the currently detected target is matched with the target positions in each frame image of the consecutive frames adjacent to the frame image where the current detected target is located, multiple trajectory matches will be generated, resulting in low efficiency in determining the trajectory set. At the same time, each trajectory in the trajectory set is the trajectory of the corresponding same target. Therefore, the embodiment of the present invention performs trajectory matching on the current detected target with the tail point target of each trajectory in the trajectory set.

[0073] At the same time, taking into account the situation mentioned above that misjudgment may occur in two frames of images that are far apart in time, the present invention adds the judgment of the number of interval frames between the image where the target at the end of each trajectory is located and the frame image where the current detection target is located during trajectory matching.

[0074] Specifically, each detection target in each frame image is traversed sequentially according to the order of each frame image in the monitoring image sequence. During the traversal process, the detection target in the current frame image is tracked by the target position of the tail target of each trajectory in the trajectory set, combined with the number of frames between the frame image where the tail target in each trajectory is located and the current frame image. If the match is successful, the detection target is added to the corresponding trajectory as the new tail target of the trajectory. If the match fails, the detection target is added to the trajectory set as a new trajectory. In addition, when tracking the detection target in the current frame image, if the trajectory set is an empty set, the detection target is directly added as a new trajectory to the trajectory set. After the above operation is completed, the next frame image of the current frame is used as the current frame image, and subsequent traversal operations are performed until the trajectory matching operation for each frame image in the monitoring image sequence is completed. At this time, the traversal ends and the trajectory set of the monitoring image sequence is obtained. The number of frames of interval is pre-set.

[0075] It should be noted that each target stored in each trajectory in the trajectory set can be shown as the information of the detection frame of each detection target. When the number of frames between the current frame image and the frame image where the target of each trajectory end point is located meets the preset frame number condition, the trajectory matching can be performed by the horizontal and vertical offsets corresponding to the horizontal and vertical coordinates of the center point of the target position of the detection target in the current frame image and the horizontal and vertical coordinates of the center point of the position of each trajectory end point target in the trajectory set, as well as the distance between the center point of the target position of the detection target in the current frame image and the center point of the position of each trajectory end point target in the trajectory set. The trajectory matching can also be performed by the overlapping area (intersection-over-union ratio) of the detection target in the current frame image and the target of each trajectory end point in the trajectory set. The embodiment of the present invention does not impose any restrictions on this. When the number of frames between the current frame image and the frame image where the target of each trajectory end point is located does not meet the preset frame number condition, the detection target is directly added to the trajectory set as a new trajectory.

[0076] Based on the above embodiment, step 121 includes:

[0077] Track matching is performed on the detected target in the current frame image based on the target position of the detected target in the current frame image of the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of each trajectory in the trajectory set. The extension direction is determined based on the sequence and positional relationship of the targets in the corresponding trajectory.

[0078] Taking into account that the rotation direction of the monitored object in the monitoring image sequence is fixed, it can be known that the trajectory of the same detection target in the monitoring image sequence must extend along one direction. Therefore, the embodiment of the present invention adds the judgment of the trajectory extension direction when performing trajectory matching on the detection target in the current frame image.

[0079] Specifically, the detected target in the traversed current frame image is matched with the target position of the tail point target of each trajectory in the trajectory set, combined with the interval frame number between the frame image where the tail point target in each trajectory is located and the current frame image and the extension direction of each trajectory, to perform trajectory matching on the detected target in the current frame image.

[0080] It should be noted that the extension direction of each trajectory is determined by the order of the frames in which the targets appear within each trajectory, as well as the positional relationship between the targets. For example, if the image originates from the upper left corner and its horizontal coordinate increases toward the right, and the monitored object rotates clockwise along its central axis, with its central axis perpendicular to the horizontal plane, then the horizontal coordinate of the target position center point in the subsequent frame of the same detected target in the monitoring image sequence must be greater than the horizontal coordinate of the target position center point in the previous frame. In this case, the extension direction of the target's trajectory is to the right.

[0081] Based on the above embodiments, Figure 3 FIG. 1 is a flow chart of the trajectory matching method provided by the present invention. Figure 3 As shown, the steps are based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of the trajectory, and the detection target in the current frame image is tracked, including:

[0082] Step 310 , determining a position matching result of the detected target in the current frame image based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory;

[0083] Specifically, according to the width and height information of the target position of the detection target in the current frame image and the width and height information of the target position of the tail point target in each track in the track set, the position matching result of the detection target in the current frame image and the tail point target in each track is obtained. Among them, the width and height information is the vertical coordinate (y top ) and the vertical coordinate of the lower right vertex (y bottom ) indicates the high information of the detected target, and the horizontal coordinate (x top ) and the horizontal coordinate of the lower right vertex (x bottom ) represents the wide information of the detected target.

[0084] It should be noted that the position matching result of the detection target in the current frame image can be determined by calculating the intersection-and-union ratio of the detection target area in the current frame image and the tail point target area in each track in the track set through the width and height information, and classifying based on the intersection-and-union ratio of the tail point target area in each obtained track; it can also be determined by calculating the intersection-and-union ratio of the width and the intersection-and-union ratio of the height of the detection target in the current frame image and the tail point target in each track in the track set through the width and height information, and classifying based on the intersection-and-union ratio of the width and the intersection-and-union ratio of the height of the tail point target in each obtained track. The embodiment of the present invention does not impose any restrictions on this.

[0085] Step 320 , based on the center position of the target position of the detected target in the current frame image, determining the estimated extension direction of the detected target in the current frame image and the tail point in each trajectory, and determining the trajectory consistency status based on the extension direction of each trajectory and the estimated extension direction;

[0086] Specifically, the coordinates of the center point of the target position of the detection target in the current frame image and the coordinates of the center point of the target position of the tail point target in each trajectory in the trajectory set are used to determine the estimated extension direction of the tail point target of each trajectory to the monitoring target in the current frame image. The estimated extension direction of the tail point target of each trajectory to the monitoring target in the current frame image is compared with the extension direction of each trajectory to determine the trajectory consistency state of the monitoring target in the current frame image and each trajectory. It should be noted that if the estimated extension direction and the trajectory extension direction are the same, the trajectory consistency state indicates consistency; otherwise, the trajectory consistency state indicates inconsistency.

[0087] Step 330, determining the temporal continuity state based on the number of frames between the current frame image and the frame image where the tail target in each trajectory is located;

[0088] Specifically, the temporal continuity between the current frame image and the frame image containing the tail target in each trajectory is determined based on whether the frame number interval between the current frame image and the frame image containing the tail target in each trajectory meets a preset frame number condition. It should be noted that the temporal continuity state can be continuous if the frame number interval is less than the preset frame number, and discontinuous otherwise.

[0089] Here, step 310, step 320 and step 330 may be executed in parallel or in a sequential order, which is not limited in the embodiment of the present invention.

[0090] Step 340 : performing trajectory matching on the detected target in the current frame image based on the position matching result, the trajectory consistency state, and the time continuity state.

[0091] It should be noted that, when performing trajectory matching on the detection target in the current frame image, the detection target in the current frame image and the tail point target of each trajectory in the trajectory set can be first position-matched. If the position matching result is a successful match, that is, the detection target in the current frame image belongs to one of the existing trajectories in the trajectory set, then the trajectory consistency state and the time continuity state are judged; the trajectory consistency state and the time continuity state can also be judged first. If the trajectory consistency state is judged to be consistent and the time continuity state is judged to be continuous, the detection target in the current frame image is position-matched with the tail point target of each trajectory in the judgment result. The embodiment of the present invention does not impose any restrictions on this.

[0092] Based on the above embodiments, Figure 4 FIG. 1 is a flow chart of the position matching method provided by the present invention. Figure 4 As shown, step 310 includes:

[0093] Step 311, based on the width and height information of the target position of the detection target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory, determine the height intersection and width intersection and width intersection ratio of the detection target in the current frame image and the tail point target in each trajectory;

[0094] Specifically, the width information of the target position of the detected target in the current frame image and the width information of the target position information of the tail point target in the trajectory are calculated by the intersection and union of width, and the height information of the target position of the detected target in the current frame image and the height information of the target position information of the tail point target in the trajectory are calculated by the intersection and union of height. The specific calculation formula is:

[0095]

[0096]

[0097]

[0098]

[0099] Where i represents the i-th detected target in the current frame, j represents the tail point target of the j-th track in the track set, is the height intersection-over-union ratio of the i-th detected target in the current frame and the tail point target of the j-th track in the track set, is the intersection-over-union ratio of the width of the i-th detected target in the current frame and the tail point target of the j-th track in the track set, It is the length of the intersection of the i-th detected target in the current frame and the tail target height of the j-th track in the track set. It can be a negative number. When it is a negative number, it means there is no intersection. It is the length of the intersection of the width of the i-th detected target in the current frame and the tail point target of the j-th track in the track set. It can be a negative number. When it is a negative number, it means there is no intersection. Represents the coordinates of the lower right vertex of the detection box of the i-th detection target in the current frame, Represents the coordinates of the upper left vertex of the detection box of the i-th detection target in the current frame, Represents the coordinates of the lower right vertex of the detection box of the tail target of the j-th track in the track set, Represents the coordinate of the upper left vertex of the detection box of the tail target of the j-th track in the track set.

[0100] Step 312: determining a matching loss value between the detected target in the current frame image and the tail point target in each trajectory based on the height intersection-over-union ratio and the width intersection-over-union ratio of the detected target in the current frame image and the tail point target in each trajectory;

[0101] It should be noted that, based on the formula in step 310, the calculation formula for the matching loss value between the detected target in the current frame image and the tail point target in each trajectory is as follows:

[0102]

[0103]

[0104]

[0105] In the formula, λ is the weight value, which ranges from 0 to 1, and cost i,j Represents the matching loss value between the i-th detected target in the current frame and the tail point target of the j-th track in the track set.

[0106] Step 313: Determine the position matching result based on the matching loss value between the detected target in the current frame image and the tail point target in each trajectory.

[0107] It should be noted that, according to the matching loss value of the detection target in the current frame image and the tail point target in each trajectory obtained in step 312, the position matching result can be determined by sorting the matching loss values ​​of the detection target in the current frame image and the tail point target in each trajectory, and taking the minimum value as the best matching trajectory. It is also possible to input the matching loss value of the detection target in the current frame image and the tail point target in each trajectory into a matching classification algorithm, such as the Hungarian algorithm, to obtain the position matching result output by the matching algorithm. The embodiment of the present invention does not limit this.

[0108] Based on the above embodiments, Figure 5 FIG. 1 is a flow chart of the method for obtaining target recognition results provided by the present invention. Figure 5 As shown, step 130 includes:

[0109] Step 131 , performing intersection-over-union calculations on any track in the track set of the monitoring image sequence and each track in the template track set, to obtain a matching loss value between the track and each track in the template track set;

[0110] Specifically, first, each track in the track set of the monitoring image sequence and each track in the template track set are uniformly aligned so that the number of targets in each track in the track set of the monitoring image sequence and each track in the template track set is the same. Then, each aligned track in the track set of the monitoring image sequence is calculated with each track in the template track set to obtain the matching loss value of the track and each track in the template track set, where the matching loss value represents the degree of overlap between the two tracks, and the value is between 0 and 1. The closer the matching loss value is to 0, the higher the degree of overlap between the two tracks. The intersection and union ratio of the track and a track in the template track set is calculated as follows:

[0111] First, the intersection-and-union ratio is calculated for each target in the aligned trajectory and the target with the same index position in the trajectory of the template trajectory set to obtain the matching loss value of the target. Then, the matching loss values ​​of each target in the trajectory are averaged to obtain the matching loss value of the trajectory with the trajectory in the template trajectory set.

[0112] Step 132 : Based on the matching loss values ​​of each track in the track set of the monitoring image sequence and each track in the template track set, the number of unmatched tracks is determined as the target recognition result.

[0113] Specifically, the matching loss values ​​of each track in the track set of the monitoring image sequence and each track in the template track set are input into a matching classification algorithm, such as the Hungarian algorithm, to obtain the matching result output by the matching classification algorithm, and the number of unsuccessful matching tracks in the matching result is used as the recognition result.

[0114] Based on the above embodiments, Figure 6 Schematic diagram of the process of obtaining a template set provided by the present invention. Figure 6 As shown, the template trajectory set in step 130 is determined based on the following steps:

[0115] Step 610, determining multiple groups of initial image sequences;

[0116] Step 620 , determining a trajectory set of any initial image sequence based on the target positions of each detection target in a plurality of consecutive frames of images in any initial image sequence in the plurality of groups of initial image sequences;

[0117] Taking into account multiple sets of initial image sequences, a more comprehensive and accurate template trajectory set can be obtained. Therefore, the embodiment of the present invention generates a template trajectory set through multiple sets of initial image sequences.

[0118] Specifically, a trajectory set is generated for each of the multiple initial image sequences. The trajectory set is generated by using any of the above trajectory set generation methods to obtain multiple trajectory sets corresponding to the multiple initial image sequences.

[0119] Step 630 , calculating the intersection-over-union ratios of each track in the track set of each initial image sequence in the multiple groups of initial image sequences, and obtaining matching values ​​between each track;

[0120] Specifically, the trajectories in the multiple trajectory sets obtained in step 620 are aligned uniformly so that the number of targets in each trajectory set is the same. Then, the intersection and union ratios of each trajectory in each trajectory set are calculated with the trajectories in the other trajectory sets to obtain the matching values ​​between each two trajectories. The intersection and union ratios of each two trajectories are calculated as follows:

[0121] First, the intersection-and-union (IoU) calculation is performed on each target in one of the two trajectories and the target with the same index position in the other trajectory to obtain the matching value of the target. Then, the matching values ​​of each target in the aligned trajectory are averaged to obtain the matching values ​​of the two trajectories.

[0122] Step 640: Determine a template trajectory set based on the matching values ​​between each trajectory and a preset threshold.

[0123] Specifically, if the matching value of two trajectories is greater than a preset threshold, the matching loss value is calculated based on the matching value, and the matching loss value between each pair of trajectories is input into the matching classification algorithm, such as the Hungarian algorithm, to obtain the matching result output by the matching classification algorithm. If the two trajectories in the matching result are the same, the two trajectories are fused, and based on this rule, the template trajectory set is determined.

[0124] Based on the above embodiments, an embodiment of the present invention also provides a target recognition method. The method takes a loom as an example, but is not limited to other similar scenarios. The method is divided into two stages: the first stage is the stage of generating a template trajectory set and the second stage is the stage of target recognition in the monitoring image sequence.

[0125] The multiple initial image sequences and monitoring image sequences in both phases are generated by the loom monitoring camera. Each rotation of the loom, and therefore the fabric, produces a cycle of image frames captured by the camera. This is known as the initial image sequence or monitoring image sequence. Each frame in the sequence captures the appearance of the loom at a different rotation angle, i.e., the local state of the fabric from different viewing angles. A complete cycle of image sequences records the state of the entire fabric. Figure 7 FIG. 1 is a top view schematic diagram of the rotation of the object to be monitored provided by the present invention. Figure 7 As shown, the outer frame is the surface of the object to be monitored, a starting mark is engraved on the surface, the direction of the arrow is the rotation direction of the object to be monitored, the triangle is the camera, and the two dotted arrows at the vertices of the triangle are the maximum angles that the camera can capture.

[0126] Figure 8 This is a schematic diagram of the template trajectory set generation stage provided by the present invention. Figure 8 As shown, the first stage (generating template trajectory set stage):

[0127] The initial image sequences of K consecutive cycles are sent to the target detector respectively. The target detector is mainly responsible for extracting the detection targets in each frame image of each group of initial image sequences and generating recognition results, that is, template trajectory sets. Figure 9 Schematic diagram of the structure of the target detector provided by the present invention. Figure 9 As shown in Figure 1, the target detector consists of an encoder, a positioning identifier, and an inter-frame matcher.

[0128] in,

[0129] The encoder consists of multiple layers of convolution, which is used to extract frame image features and output a representation map of a specified dimension. The positioning identifier consists of multiple layers of convolution and fully connected layers, which is used to output the target position and category of the detected target in the current frame, that is, the horizontal coordinate x and vertical coordinate y of the center of the detection box, the width w and height h of the target area, and the category c. The inter-frame matcher is composed of a weighted Hungarian algorithm, which is used to perform serial matching on the detection results of each frame to eliminate false alarms and identify the real target.

[0130] Figure 10 This is a schematic diagram of the inter-frame matching process provided by the present invention. Figure 10As shown in the figure, the processing flow of the inter-frame matcher is as follows: First, the trajectory set is initialized (empty), where the trajectory set is used to record the detection box information of each detection target with spatiotemporal continuity. Each trajectory is a series of detection boxes that are successfully matched in series. Then, the result is compared with the trajectory set frame by frame. If the trajectory set is empty, the detection box of the current detection target in the current frame is used as a new trajectory and updated to the trajectory set. If the trajectory set is not empty, the target position (detection box information) of the detection target in the current frame is calculated with the target position of the tail target of each trajectory in the trajectory set. The position matching result is then output using the Hungarian algorithm. The position matching result is further restricted to eliminate those matching results that are far apart in time and space: matching results are eliminated based on the temporal continuity state, including requiring the frame difference before and after the match to be no more than 3 frames, that is, a maximum of 3 frames can be lost; and based on the spatial continuity (trajectory consistency state), including the center of the target position of the detection target in the current frame should move in the same direction as the cloth rotates. When rotating counterclockwise, the new detection center position should be larger than the previous detection center position. The time continuity state and trajectory consistency state are used as threshold values. If the threshold value is true at the same time as the position matching, the match is considered successful. The detection box of the current detection target is spliced ​​onto the corresponding trajectory to update the trajectory. If the match is unsuccessful, the detection box of the current detection target is used as the new trajectory and updated to the trajectory set. Finally, based on the trajectory length (the number of targets in the trajectory), tracks less than a specific threshold are deleted. For example, if it is set to 5, only tracks with longer recognition duration are retained, and the trajectory set of the current initial image sequence is output. The number of tracks in the trajectory set represents the number of valid targets. The matching loss value is calculated by the following formula:

[0131]

[0132]

[0133]

[0134]

[0135]

[0136]

[0137] Where,

[0138] i represents the i-th detected target in the current frame, j represents the tail point target of the j-th track in the track set, It is the length of the intersection of the i-th detected target in the current frame and the tail target height of the j-th track in the track set. It can be a negative number. When it is a negative number, it means there is no intersection. It is the length of the intersection of the width of the i-th detected target in the current frame and the tail point target of the j-th track in the track set. It can be a negative number. When it is a negative number, it means there is no intersection. Represents the coordinates of the lower right vertex of the detection box of the i-th detection target in the current frame, Represents the coordinates of the upper left vertex of the detection box of the i-th detection target in the current frame, Represents the coordinates of the lower right vertex of the detection box of the tail target of the j-th track in the track set, Represents the coordinates of the upper left vertex of the detection box of the tail target of the j-th track in the track set, Represents the horizontal coordinate of the center point of the target position of the i-th detected target in the current frame, The horizontal coordinate of the center point of the target position of the tail point target of the jth trajectory in the trajectory set, t i Indicates the index position of the frame image where the i-th detection target in the current frame is located in the image sequence; t j Indicates the index position of the frame image where the tail target of the jth trajectory in the trajectory set is located in the image sequence, λ is the weight value, ranging from 0 to 1, and cost i,j Indicates the matching loss value of the i-th detection target in the current frame and the tail point target of the j-th track in the track set, gate i,j Indicates the threshold value of the i-th detected target in the current frame and the tail point target of the j-th track in the track set.

[0139] After obtaining multiple sets of trajectory sets corresponding to multiple sets of initial image sequences, the time intervals of the multiple trajectory sets are normalized to align sequences of unequal lengths. The results are then fused to obtain a template trajectory set. Considering that the initial image sequences of each cycle have the same starting position and are sampled at equal intervals over one rotation of the loom, T frames represent T intervals. These are uniformly normalized and aligned to 120 intervals in the temporal dimension, and linear interpolation is used to complete the recognition results of the missing intervals. Specifically, each trajectory set is first divided into four point sets based on the four vertices of the detection box: upper left, lower left, upper right, and lower right. Each point set consists of a series of two-dimensional data points, such as {[t1,x1],[t2,x2],[t3,x3],…} left_top ,[t i ,x i ] i Indicates that the detection box is in the i-th frame, x i"i" represents the coordinates of the top-left vertex of the detection box in the trajectory. i is a natural number. New interval points are calculated using linear interpolation. If a trajectory set contains N targets, then N targets remain after normalization. The union of the detected regions of each normalized trajectory set is then taken interval by interval to generate the fused recognition result, i.e., the template trajectory set. If there is significant overlap in the corresponding regions during fusion (i.e., if the Intersection-Union Ratio is greater than 0.8), they are considered the same target; otherwise, they are new targets. Therefore, the fused recognition result contains at most M targets, where M = max{n1,n2,n3,…}, where n is the number of identified targets in each cycle.

[0140] Figure 11 Schematic diagram of the target recognition stage in the monitoring image sequence provided by the present invention. Figure 11 As shown in the second stage (target recognition stage in the monitoring image sequence):

[0141] The monitoring image sequence of the current cycle to be queried is sent to the target detector to extract the trajectory set. The process is the same as the circle-by-circle processing in the first stage of generating the template trajectory set. The trajectory set is normalized with the time interval mentioned above and aligned with the space in the template trajectory set. Finally, the matching value and matching loss value of the trajectories in the aligned trajectory set are calculated one by one with the trajectories in the template trajectory set. The target matching is completed using the Hungarian algorithm. At the same time, the matching results with low overlap ratios (i.e., matching values) and those that cannot be a group are filtered by the threshold value to obtain the target recognition result. Among them, the matching value uses the dice indicator, that is, the intersection-over-union ratio of the 3D area. The specific formula is as follows:

[0142] cost i,j =1-dice i,j

[0143]

[0144] Where,

[0145] i represents the i-th track in the track set of the monitoring image sequence, j represents the j-th track in the template track set, and cost i,j represents the matching loss value of the i-th track in the track set of the monitoring image sequence and the j-th track in the template track set, dice i,j Indicates the matching value between the i-th track in the track set of the monitoring image sequence and the j-th track in the template track set, gate i,j Represents the threshold value of the i-th track in the track set of the monitoring image sequence and the j-th track in the template track set.

[0146] The number of unmatched targets in the target recognition results determines the fabric state, that is, the recognition result of whether there are defects. If the number of remaining unmatched targets is greater than 0, the "Fabric Defect" label is output; otherwise, the "Fabric Normal" label is output.

[0147] It should be noted that the target detector contains learnable parameters, and the iterative training process of its encoder and positioning identifier includes: the recognition and positioning loss function is composed of the regression position mean square error loss superimposed on the classification cross entropy loss. After the output is obtained through forward propagation, the gradient is calculated according to the combined loss function, feedback training is performed, and it is iterated until convergence.

[0148] The forward test after training is: the second stage mentioned above (the target recognition stage in the monitoring image sequence).

[0149] The target recognition device provided by the present invention is described below. The target recognition device described below and the target recognition method described above can be referenced to each other.

[0150] Figure 12 The schematic diagram of the structure of the target recognition device provided by the present invention. Figure 12 As shown, the voice interaction device includes: a determination module 1210 , a trajectory generation module 1220 and a recognition module 1230 .

[0151] in,

[0152] The determination module 1210 is used to determine a monitoring image sequence, perform target recognition on each frame image in the monitoring image sequence, and obtain the target position of the detection target in each frame image;

[0153] The trajectory generation module 1220 is used to determine a trajectory set of the monitoring image sequence based on the target position of each detection target in a plurality of consecutive frames of the monitoring image sequence;

[0154] The recognition module 1230 is used to compare the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result. The template trajectory set is determined based on the target position of each initial target in the continuous multiple frames of the initial image sequence.

[0155] In an embodiment of the present invention, a determination module 1210 is used to determine a monitoring image sequence, perform target recognition on each frame image in the monitoring image sequence, and obtain the target position of the detection target in each frame image; a trajectory generation module 1220 is used to determine a trajectory set of the monitoring image sequence based on the target position of each detection target in multiple consecutive frames of images in the monitoring image sequence; an identification module 1230 is used to compare the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result. The template trajectory set is determined based on the target position of each initial target in multiple consecutive frames of images in the initial image sequence, thereby realizing dynamic comparative recognition of the detection target trajectory of the monitored object and the template trajectory, thereby improving the accuracy of detection of each detection target and thereby improving the recognition effect of the detection target.

[0156] Based on any of the above embodiments, the trajectory generation module 1220 includes:

[0157] Trajectory matching submodule: It is used to perform trajectory matching on the detected target in the current frame image based on the target position of the detected target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of frames between the frame image where the tail point target in each trajectory is located and the current frame image;

[0158] The judgment submodule is used to add the successfully matched detection targets to the corresponding trajectory, build a new trajectory based on the detection targets that failed to match, and update the next frame image of the current frame image to the current frame image.

[0159] Based on any of the above embodiments, the trajectory matching submodule is further used to perform trajectory matching on the detection target in the current frame image based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of each trajectory in the trajectory set; the extension direction is determined based on the sequence and positional relationship of the targets in the corresponding trajectory.

[0160] Based on any of the above embodiments, the trajectory matching submodule includes:

[0161] A position matching submodule is used to determine a position matching result of the detection target in the current frame image based on the width and height information of the target position of the detection target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory;

[0162] A consistency judgment submodule is used to determine the estimated extension direction of the detected target in the current frame image and the tail point of each track based on the center position of the target position of the detected target in the current frame image, and determine the track consistency status based on the extension direction of each track and the estimated extension direction;

[0163] The continuity judgment submodule is used to determine the temporal continuity state based on the interval between the current frame image and the frame image of the tail target in each trajectory;

[0164] The current detection target trajectory matching submodule is used to perform trajectory matching on the detection target in the current frame image based on the position matching result, trajectory consistency state and time continuity state.

[0165] Based on any of the above embodiments, the position matching submodule includes:

[0166] A submodule for calculating the aspect ratio intersection-and-union (IoU) is used to determine the IoU of height and the IoU of width of the detected target in the current frame image and the tail point target in each trajectory based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory;

[0167] A matching loss value calculation submodule is used to determine the matching loss value between the detection target in the current frame image and the tail point target in each trajectory based on the height intersection and width intersection ratio of the detection target in the current frame image and the tail point target in each trajectory;

[0168] The position matching result calculation submodule is used to determine the position matching result based on the matching loss value of the detected target in the current frame image and the tail point target in each trajectory.

[0169] Based on any of the above embodiments, the identification module 1230 includes:

[0170] The trajectory matching loss value calculation submodule is used to calculate the intersection-and-union ratio of any trajectory in the trajectory set of the monitoring image sequence and each trajectory in the template trajectory set, and obtain the matching loss value of the trajectory and each trajectory in the template trajectory set;

[0171] The recognition result calculation submodule is used to determine the number of unmatched trajectories as the target recognition result based on the matching loss value of each trajectory in the trajectory set of the monitoring image sequence and each trajectory in the template trajectory set.

[0172] Based on any of the above embodiments, the identification module 1230 further includes:

[0173] An initial image sequence determination submodule is used to determine multiple groups of initial image sequences;

[0174] a trajectory set determination submodule, configured to determine a trajectory set of an initial image sequence based on a target position of each detection target in a plurality of consecutive frames of images in any initial image sequence of the plurality of initial image sequences;

[0175] A matching value calculation submodule is used to calculate the intersection-and-union ratio of each track in the track set of each initial image sequence in the multiple groups of initial image sequences, and obtain the matching value between each track;

[0176] The template trajectory set determination submodule is used to determine the template trajectory set based on the matching value between each trajectory and a preset threshold.

[0177] Figure 13 An example of a physical structure diagram of an electronic device is shown below. Figure 13 As shown, the electronic device may include: a processor 1310, a communication interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communication interface 1320, and the memory 1330 communicate with each other via the communication bus 1340. The processor 1310 may call logic instructions in the memory 1330 to execute a target recognition method, which includes: determining a monitoring image sequence, performing target recognition on each frame image in the monitoring image sequence, and obtaining a target position of a detected target in each frame image; determining a trajectory set of the monitoring image sequence based on the target position of each detected target in a plurality of consecutive frames of the monitoring image sequence; and comparing a template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result, wherein the template trajectory set is determined based on the target position of each initial target in a plurality of consecutive frames of the initial image sequence.

[0178] In addition, the logic instructions in the above-mentioned memory 1330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0179] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target recognition method provided by the above-mentioned methods, which includes: determining a monitoring image sequence, performing target recognition on each frame image in the monitoring image sequence, and obtaining the target position of the detection target in each frame image; determining a trajectory set of the monitoring image sequence based on the target position of each detection target in multiple consecutive frames of images in the monitoring image sequence; comparing the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result, and the template trajectory set is determined based on the target position of each initial target in multiple consecutive frames of images in the initial image sequence.

[0180] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the target recognition method provided by the above-mentioned methods, the method comprising: determining a monitoring image sequence, performing target recognition on each frame image in the monitoring image sequence, and obtaining the target position of the detection target in each frame image; determining a trajectory set of the monitoring image sequence based on the target position of each detection target in multiple consecutive frames of images in the monitoring image sequence; comparing the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result, the template trajectory set being determined based on the target position of each initial target in multiple consecutive frames of images in the initial image sequence.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0182] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A target recognition method, characterized in that: include: Determine a monitoring image sequence, perform target recognition on each frame image in the monitoring image sequence, and obtain a target position of a detection target in each frame image; The monitoring image sequence is a sequence of consecutive frames of the same size in the same direction, each frame containing the entire monitored object at a different angle. The monitored object is a periodically rotating object, and a set of monitoring image sequences is generated for each rotation of the monitored object. When the monitored object rotates, different detection targets adjacent to the monitored object have different trajectories in the monitoring image sequence generated by each rotation of the monitored object. determining a trajectory set of the monitoring image sequence based on a target position of each detection target in a plurality of consecutive frames of images in the monitoring image sequence; The template trajectory set is compared with the trajectory set of the monitoring image sequence to obtain a target recognition result, wherein the template trajectory set is determined based on the target position of each initial target in the continuous multiple frames of the initial image sequence.

2. The target recognition method according to claim 1, characterized in that: The determining of the trajectory set of the monitoring image sequence based on the target position of each detection target in the continuous multiple frames of the monitoring image sequence includes: performing trajectory matching on the detected target in the current frame image based on the target position of the detected target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image; The successfully matched detection targets are added to the corresponding tracks, a new track is constructed based on the detection targets that failed to match, and the next frame image of the current frame image is updated as the current frame image.

3. The target recognition method according to claim 2, characterized in that: The performing trajectory matching on the detection target in the current frame image based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, and the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image includes: Based on the target position of the detection target in the current frame image of the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of each trajectory in the trajectory set, the detection target in the current frame image is tracked and matched; the extension direction is determined based on the sequence and positional relationship of each target in the corresponding trajectory.

4. The target recognition method according to claim 3, characterized in that: The method of performing trajectory matching on the detection target in the current frame image based on the target position of the detection target in the current frame image in the monitoring image sequence, the target position of the tail point target in each trajectory in the trajectory set, the number of interval frames between the frame image where the tail point target in each trajectory is located and the current frame image, and the extension direction of the trajectory includes: Determining a position matching result of the detected target in the current frame image based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory; determining, based on a center position of a target position of a detection target in the current frame image, an estimated extension direction of the detection target in the current frame image and an end point in each trajectory, and determining a trajectory consistency state based on the extension direction of each trajectory and the estimated extension direction; Determining a temporal continuity state based on the number of frames between the current frame image and the frame image where the tail point target in each trajectory is located; Based on the position matching result, the trajectory consistency state and the time continuity state, trajectory matching is performed on the detection target in the current frame image.

5. The target recognition method according to claim 4, characterized in that: The determining a position matching result of the detected target in the current frame image based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory includes: Determining a height intersection-and-joint ratio and a width intersection-and-joint ratio of the detected target in the current frame image and the tail point target in each trajectory based on the width and height information of the target position of the detected target in the current frame image and the width and height information of the target position information of the tail point target in each trajectory; Determining a matching loss value between the detected target in the current frame image and the tail point target in each trajectory based on an intersection-over-union ratio of heights and an intersection-over-union ratio of widths; The position matching result is determined based on the matching loss value of the detected target in the current frame image and the tail point target in each trajectory.

6. The target recognition method according to any one of claims 1 to 5, characterized in that: The comparing the template trajectory set with the trajectory set of the monitoring image sequence to obtain a target recognition result includes: Calculate the intersection-and-union ratio of any trajectory in the trajectory set of the monitoring image sequence and each trajectory in the template trajectory set, and obtain a matching loss value between the any trajectory and each trajectory in the template trajectory set; Based on the matching loss value of each track in the track set of the monitoring image sequence and each track in the template track set, the number of unsuccessfully matched tracks is determined as the target recognition result.

7. The target recognition method according to any one of claims 1 to 5, characterized in that: The template trajectory set is determined based on the following steps: determining a plurality of groups of initial image sequences; determining a trajectory set of any initial image sequence based on target positions of each detection target in a plurality of consecutive frames of images in any initial image sequence in the plurality of groups of initial image sequences; calculating an intersection-over-union ratio for each track in the track set of each initial image sequence in the plurality of groups of initial image sequences, and obtaining a matching value between each track; The template trajectory set is determined based on the matching values ​​between each trajectory and a preset threshold.

8. A target recognition device, characterized in that: include: a determination module configured to determine a monitoring image sequence, perform target recognition on each frame of the monitoring image sequence, and obtain a target position of a detection target in each frame of the image; the monitoring image sequence is a sequence of consecutive frames of the same orientation and size, each frame containing the entire monitored object at a different angle; the monitored object is a periodically rotating object, and a set of monitoring image sequences is generated for each rotation of the monitored object; wherein, when the monitored object rotates, different detection targets adjacent to the monitored object have different trajectories in the monitoring image sequence generated by each rotation of the monitored object; a trajectory generation module, configured to determine a trajectory set of the monitoring image sequence based on a target position of each detection target in a plurality of consecutive frames of the monitoring image sequence; The recognition module is used to compare a template trajectory set with a trajectory set of the monitoring image sequence to obtain a target recognition result, wherein the template trajectory set is determined based on the target position of each initial target in a plurality of consecutive frames of images in the initial image sequence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the target recognition method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target recognition method according to any one of claims 1 to 7 are implemented.

Images