Image screening method, device, equipment and storage medium

Abstract

This disclosure provides an image screening method, apparatus, device, and storage medium. The method includes: acquiring at least two detection images containing an object to be detected; determining frequency information for each detection image; wherein the frequency information is used to characterize the information changes of pixels in the detection image; acquiring a frequency set of at least two frequency bands corresponding to each detection image from the frequency information of each detection image; determining the similarity between adjacent detection images based on the frequency sets of at least two frequency bands corresponding to each of the at least two detection images; and acquiring a target detection image of the object to be detected from the at least two detection images based on the similarity between the adjacent detection images.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer technology, and in particular to an image screening method, apparatus, device, and storage medium. Background Technology

[0002] In scenarios such as pedestrian re-identification, cameras typically capture a dozen frames per second. Over the tens of seconds a pedestrian appears, hundreds or even thousands of images can be captured. If every image of every pedestrian were processed and saved, the storage, computing, and bandwidth requirements of the entire system would be substantial. Summary of the Invention

[0003] In view of the above, the present disclosure provides at least one image screening method, apparatus, device, and storage medium.

[0004] The technical solution of this disclosure embodiment is implemented as follows:

[0005] On one hand, embodiments of this disclosure provide an image filtering method, comprising: acquiring at least two detection images containing an object to be detected; determining frequency information of each detection image; wherein the frequency information is used to characterize the information changes of pixels in the detection image; acquiring a frequency set of at least two frequency bands corresponding to each detection image from the frequency information of each detection image; determining the similarity between adjacent detection images based on the frequency sets of at least two frequency bands corresponding to each of the at least two detection images; and acquiring a target detection image of the object to be detected from the at least two detection images based on the similarity between the adjacent detection images.

[0006] On the other hand, embodiments of this disclosure provide an image filtering device, comprising: a first acquisition module, configured to acquire at least two frames of detection images containing an object to be detected; a first determination module, configured to determine frequency information of each of the detection images; wherein the frequency information is used to characterize the information change of pixels in the detection images; a second acquisition module, configured to acquire a frequency set of at least two frequency bands corresponding to each of the detection images from the frequency information of each of the detection images; a second determination module, configured to determine the similarity between adjacent frame detection images based on the frequency sets of at least two frequency bands corresponding to each of the at least two frame detection images; and a third acquisition module, configured to acquire a target detection image of the object to be detected from the at least two frame detection images based on the similarity between the adjacent frame detection images.

[0007] In another aspect, embodiments of this disclosure provide a computer device including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0008] In another aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this disclosure provide a computer program including computer-readable code, which, when executed in a computer device, causes a processor in the computer device to perform some or all of the steps in the above-described method.

[0010] In another aspect, embodiments of this disclosure provide a computer program product, the computer program product including a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a computer, it implements some or all of the steps in the above method.

[0011] In this embodiment, firstly, by acquiring at least two detection images containing the object to be detected, the frequency information of each detection image is accurately determined; wherein, the frequency information is used to characterize the information changes of pixels in the detection image. Then, the frequency set of at least two frequency bands corresponding to each detection image is obtained from the frequency information of each detection image; based on the frequency sets of at least two frequency bands corresponding to adjacent detection images in the at least two detection images, the similarity between adjacent detection images is quickly and accurately determined. Finally, based on the similarity between adjacent detection images, the target detection image of the object to be detected is simply and accurately obtained from the at least two detection images. In this way, by selecting representative target detection images to characterize the overall action of the object to be detected, significant resources can be saved during feature calculation and storage of the object to be detected, and the processing efficiency of storage, calculation, and transmission can be improved.

[0012] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0014] Figure 1 A schematic diagram illustrating the implementation process of an image filtering method provided in this embodiment of the disclosure;

[0015] Figure 2 A schematic diagram illustrating the implementation process of an image filtering method provided in this embodiment of the disclosure;

[0016] Figure 3 A schematic diagram illustrating the implementation process of an image filtering method provided in this embodiment of the disclosure;

[0017] Figure 4 A schematic diagram illustrating the implementation process of an image fusion method provided in this embodiment of the disclosure;

[0018] Figure 5 A schematic diagram of a discrete cosine transform provided in an embodiment of this disclosure;

[0019] Figure 6 A schematic diagram illustrating the implementation process of an image filtering method provided in this embodiment of the disclosure;

[0020] Figure 7 A schematic diagram illustrating a frequency set acquisition location provided in an embodiment of this disclosure;

[0021] Figure 8 A schematic diagram illustrating the labeling result of a detected image provided in an embodiment of this disclosure;

[0022] Figure 9 A schematic diagram of a multi-frame target detection image provided in an embodiment of this disclosure;

[0023] Figure 10 A schematic diagram of a graphical model provided in an embodiment of this disclosure;

[0024] Figure 11 This is a schematic diagram of the structure of a target fusion model provided in an embodiment of the present disclosure;

[0025] Figure 12 A schematic diagram illustrating the processing of a fusion feature provided in an embodiment of this disclosure;

[0026] Figure 13 This is a schematic diagram of the composition structure of an image filtering device provided in an embodiment of the present disclosure;

[0027] Figure 14 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0029] In the following description, references to "some embodiments" describe a subset of all possible embodiments; however, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0031] This disclosure provides an image filtering method, which can be executed by a processor of a computer device. The computer device can refer to a server, laptop computer, tablet computer, desktop computer, mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device), or any other device with image filtering capabilities. Figure 1 This is a schematic diagram illustrating the implementation process of an image filtering method provided in an embodiment of this disclosure, as shown below. Figure 1 As shown, the method includes the following steps S101 to S105:

[0032] Step S101: Obtain at least two detection images containing the object to be detected.

[0033] Here, the object to be detected can refer to an object that can be identified, such as a vehicle, animal, or pedestrian. The detection image can refer to an image containing the object to be detected, such as multiple detection images obtained by taking pictures of the object. The size of each frame of the detection image containing the object can be the same or different; for example, the size of the first detection image of the object is 50*50 pixels, and the size of the second detection image is 80*100 pixels, etc. The detection images in at least two frames containing the object have different acquisition orders; for example, in five frames of the object, the first frame was acquired before the second frame, and the second frame was acquired before the third frame, etc.

[0034] Step S101 may include: acquiring a video stream of the object to be detected; performing target recognition processing on each video image in the video stream of the object to be detected to obtain a recognition result; wherein, the recognition result includes whether the video image contains the object to be detected, and if the object to be detected is contained, the position information of the object to be detected (such as the size and position of the recognition box corresponding to the object to be detected); based on the recognition result, cropping the video image containing the object to be detected to obtain at least two detection images containing the object to be detected.

[0035] Step S102: Determine the frequency information of each of the detected images.

[0036] Here, frequency information refers to information in the frequency domain. Frequency information is used to characterize the changes in information of pixels in an image, such as the gray-level variation patterns between pixels. The frequency of an image refers to spatial frequency, which differs from the physical frequency we perceive. Therefore, to understand image frequency, we must separate the two definitions. An image can be viewed as a signal defined on a two-dimensional plane, where the amplitude of the signal corresponds to the gray level of the pixels (red, green, and blue components in a color image). If we consider only a row of pixels in a frame, it can be considered a one-dimensional signal. This signal is very similar to common time-domain signals, except that time-domain signals are defined in the time domain, while image signals are defined in the spatial domain. Therefore, the frequency of an image is called spatial frequency. Spatial frequency refers to the number of periodic changes in brightness per unit length. It reflects the spatial variation of pixel gray levels in an image. Bright spots of varying brightness can be seen on the Fourier spectrum, reflecting the degree of difference between a point and its neighborhood. For example, in an image frame, the background or slowly changing areas, where the grayscale value distribution is relatively flat, have stronger low-frequency components; while the edges, details, and noise of the image exhibit drastic spatial changes in pixel grayscale, thus representing high-frequency components.

[0037] For example, Fourier transform, Walsh transform, or cosine transform can be used to convert the detected image in the spatial domain into frequency information in the frequency domain. Different frequencies in the frequency domain can characterize different attributes of the detected image. For instance, higher frequencies can characterize the boundaries and textures of the detected image, while lower frequencies can characterize flat regions within the detected image.

[0038] Step S103: Obtain the frequency set of at least two frequency bands corresponding to each detection image from the frequency information of each detection image.

[0039] Here, a frequency set can refer to a whole that includes at least two frequency information points. For example, a frequency set might include the frequency information corresponding to all pixels in the upper left corner region of a detected image. Different frequency bands can be preset, and the frequency information of each detected image can be segmented according to these preset bands to obtain frequency sets for different frequency bands. For example, three different frequency bands can be preset, where the frequency of the first band is higher than that of the second band, which is higher than that of the third band. Here, the number and type of frequency bands corresponding to each detected image are the same. For example, the frequency sets of the first and second bands can be obtained from the frequency information of the first detected image, and the frequency sets of the first and second bands can be obtained from the frequency information of the second detected image, etc.

[0040] Step S104: Based on the frequency sets of at least two frequency bands corresponding to each of the at least two detected frames, determine the similarity between the detected frames.

[0041] Here, for all detected images, it is only necessary to determine the similarity between adjacent detected frames. For example, given five detected images of the object to be detected, determine the first similarity between the first and second detected frames, and the second similarity between the second and third detected frames, etc. The similarity between adjacent detected frames can be a numerical value used to characterize the degree of similarity between adjacent detected frames. For example, a higher similarity indicates a greater difference between adjacent detected frames. For instance: if the frequency sets of the first and second frequency bands are obtained from the frequency information of the first detected image, and the frequency sets of the first and second frequency bands are obtained from the frequency information of the second detected image; the frequency information of the first frequency band of the first detected image is subtracted from the frequency information of the first frequency band of the second detected image to obtain a first difference; the frequency information of the second frequency band of the first detected image is subtracted from the frequency information of the second frequency band of the second detected image to obtain a second difference; the first difference and the second difference are weighted and summed to obtain a weighted sum; this weighted sum is determined as the similarity between the first and second detected images.

[0042] Step S105: Based on the similarity between the adjacent frame detection images, obtain the target detection image of the object to be detected from the at least two frame detection images.

[0043] Here, the target detection image of the object to be detected can refer to an image that represents the overall behavior of the object. The target detection image can be a single frame or multiple frames. When the target detection image is multiple frames, the similarity between the target detection images is relatively high, that is, the differences between the target detection images are relatively large. A similarity threshold can be determined based on all similarities. For example, all similarities can be sorted to obtain a sorting result; a preset number of similarities from the later (i.e., lower similarity) frames can be obtained, and the average value of the preset number of similarities can be determined. This average value can then be used as the similarity threshold. If the similarity between adjacent frame detection images is greater than the similarity threshold, the target detection image of the object to be detected can be determined. For example, if the similarity threshold is 0.3, the similarity between the first and second frame detection images is 0.5, and the similarity between the second and third frame detection images is 0.2, then the second frame detection image can be determined as the target detection image.

[0044] In this embodiment, firstly, by acquiring at least two detection images containing the object to be detected, the frequency information of each detection image is accurately determined; wherein, the frequency information is used to characterize the information changes of pixels in the detection image. Then, the frequency set of at least two frequency bands corresponding to each detection image is obtained from the frequency information of each detection image; based on the frequency sets of at least two frequency bands corresponding to adjacent detection images in the at least two detection images, the similarity between adjacent detection images is quickly and accurately determined. Finally, based on the similarity between adjacent detection images, the target detection image of the object to be detected is simply and accurately obtained from the at least two detection images. In this way, by selecting representative target detection images to characterize the overall action of the object to be detected, significant resources can be saved during feature calculation and storage of the object to be detected, and the processing efficiency of storage, calculation, and transmission can be improved.

[0045] In some embodiments, step S103 may include the following steps S1031 to S1033:

[0046] Step S1031: Obtain the frequency set of the first frequency band from the upper left corner region of the transformation matrix used to characterize the frequency information.

[0047] Here, the transformation matrix is ​​obtained by performing a discrete cosine transform on the detection image and is used to characterize the frequency information of the detection image. Before step S1031, the process may further include: resizing each detection image according to a preset size to obtain a scaled detection image; performing a discrete cosine transform (DCT) on each scaled detection image to obtain a transformation matrix; and determining each transformation matrix as the frequency information of each detection image. Since the elements in the upper left corner of the transformation matrix represent higher frequency information, the elements in the upper left corner can be determined as the frequency set of the first frequency band. For example, if the size of the scaled detection image is 32*32, then the dimension of the transformation matrix is ​​32*32, and the size of the upper left corner can be 8*8 (i.e., rows 0 to 7, columns 0 to 7). The elements (64 elements) of the 8*8 upper left corner are determined as the frequency set of the first frequency band. Here, the frequency sets can be stored in the form of a list.

[0048] Step S1032: Obtain the frequency set of the second frequency band from the left and upper regions of the transformation matrix.

[0049] Here, since the elements in the upper left corner of the transformation matrix represent the highest frequency information and the elements in the lower right corner represent the lowest frequency information, the elements in the left and upper regions can be used to determine the frequency set of the second frequency band. For example, if the transformation matrix has a dimension of 32*32, the size of the left region can be 24*4 (i.e., rows 8 to 31, columns 0 to 3), and the size of the upper region can be 4*24 (i.e., rows 0 to 3, columns 8 to 31). The elements of the 24*4 left region and the elements of the 4*24 upper region (192 elements) can be used to determine the frequency set of the second frequency band.

[0050] Step S1033: Obtain the frequency set of the third frequency band from the remaining region of the transformation matrix.

[0051] Here, the remaining region is the area in the transformation matrix excluding the upper left corner, left side, and upper side regions. The frequency information of the first frequency band is higher than that of the second frequency band, and the frequency information of the second frequency band is higher than that of the third frequency band. For the frequency set of the third frequency band, the remaining region can be divided into multiple sub-regions, and the frequency information represented by one element in each sub-region can be obtained to obtain the frequency set of the third frequency band. For example, the remaining region can be divided into 4*4 sub-regions, and the element with the largest value in each 4*4 sub-region can be determined. The frequency information represented by all the largest elements (192 elements) can be determined as the frequency set of the third frequency band.

[0052] In this embodiment of the disclosure, frequency information is represented by a transformation matrix, and then the frequency information of different frequency bands can be represented by the elements at different positions of the transformation matrix, thereby quickly and accurately obtaining the frequency set of different frequency bands and improving the efficiency of image screening.

[0053] This disclosure provides an image filtering method, such as... Figure 2 As shown, the method includes the following steps S201 to S207:

[0054] Steps S201 to S203 correspond to the aforementioned steps S101 to S103 respectively, and can be implemented with reference to the specific implementation of the aforementioned steps S101 to S103; step S207 corresponds to the aforementioned step S105, and can be implemented with reference to the specific implementation of the aforementioned step S105.

[0055] Step S204: Binarize the frequency sets of at least two frequency bands corresponding to each of the adjacent frame detection images to obtain the frequency sets after frequency band binarization.

[0056] Here, the average frequency of each frequency band can be determined based on the frequency set of each frequency band. Based on the average frequency of each frequency band, the frequency set of each frequency band can be binarized to obtain the binarized frequency set of each frequency band. For example: Based on the frequency set of the first frequency band of the first frame detection image, a first frequency mean is determined, and the frequency information in the first frequency band of the first frame detection image that is greater than the first frequency mean is set to 1, and the frequency information in the first frequency band of the first frame detection image that is less than or equal to the first frequency mean is set to 0; Based on the frequency set of the second frequency band of the first frame detection image, a second frequency mean is determined, and the frequency information in the second frequency band of the first frame detection image that is greater than the second frequency mean is set to 1, and the frequency information in the second frequency band of the first frame detection image that is less than or equal to the second frequency mean is set to 0; Based on the frequency set of the first frequency band of the second frame detection image, a third frequency mean is determined, and the frequency information in the first frequency band of the second frame detection image that is greater than the third frequency mean is set to 1, and the frequency information in the first frequency band of the second frame detection image that is less than or equal to the third frequency mean is set to 0; Based on the frequency set of the second frequency band of the second frame detection image, a fourth frequency mean is determined, and the frequency information in the second frequency band of the second frame detection image that is greater than the fourth frequency mean is set to 1, and the frequency information in the second frequency band of the second frame detection image that is less than or equal to the fourth frequency mean is set to 0, and so on.

[0057] Step S205: Based on the frequency set after the frequency band is binarized, determine the sub-Hamming distance of each same frequency band in the adjacent frame detection images.

[0058] Here, Hamming distance refers to the number of substitutions required to transform one string into another. For example, the Hamming distance between 1011101 and 1001001 is 2; the Hamming distance between 2143896 and 2233796 is 3. The larger the Hamming distance, the greater the difference between adjacent frame detection images, thus defining a higher similarity between adjacent frame detection images. Sub-Hamming distance can refer to the Hamming distance determined by any set of frequencies in any frequency band. For example, the first sub-Hamming distance of the first frequency band is determined based on the binarized frequency set of the first frequency band of the first detection image and the binarized frequency set of the first frequency band of the second detection image; the second sub-Hamming distance of the second frequency band is determined based on the binarized frequency set of the second frequency band of the first detection image and the binarized frequency set of the second frequency band of the second detection image, and so on.

[0059] Step S206: The Hamming distances of all the sub-Hamming distances of the same frequency band are weighted and summed to obtain the Hamming distance between the adjacent frame detection images.

[0060] Here, the weight of each sub-Hamming distance can be customized and is not limited. For example, the weight of the first sub-Hamming distance is 1, the weight of the second sub-Hamming distance is 0.4, the weight of the third sub-Hamming distance is 0.2, etc., thereby determining the Hamming distance between the first and second detected images.

[0061] The Hamming distance between adjacent detected images is used to characterize the similarity between them. Since a larger Hamming distance indicates a greater difference between adjacent detected images, the similarity is defined as a larger Hamming distance, and the similarity (difference) between adjacent detected images is positively correlated. In some embodiments, if a negative correlation is defined between the Hamming distance and the similarity (difference), the reciprocal of the Hamming distance can be used to determine the similarity, etc.; this is not a limitation.

[0062] In this embodiment of the disclosure, by determining the sub-Hamming distances of the frequency sets after binarization of different frequency bands, and performing weighted summation on all sub-Hamming distances, the Hamming distances between adjacent frame detection images can be obtained more accurately.

[0063] This disclosure provides an image filtering method, such as... Figure 3 As shown, the method includes the following steps S301 to S306:

[0064] Steps S301 to S304 correspond to the aforementioned steps S101 to S104 respectively. When implementing these steps, you can refer to the specific implementation methods of the aforementioned steps S101 to S104.

[0065] Step S305: Determine a similarity threshold based on the Hamming distance between all the adjacent frame detection images.

[0066] Here, step S305 may include: obtaining a specified number of first Hamming distances from all Hamming distances; wherein the specified number of first Hamming distances is greater than or equal to all other Hamming distances except the specified number of first Hamming distances; determining a distance threshold based on the specified number of first Hamming distances as a similarity threshold. Since the determined similarity threshold is used to identify detection images with large differences as target detection images, and since similarity is positively correlated with the differences between adjacent frame detection images, the Hamming distances between all adjacent frame detection images can be sorted to obtain a sorting result; based on the sorting result, a specified number of Hamming distances (e.g., the top 10%) are obtained, the average Hamming distance corresponding to the specified number of Hamming distances is determined, and this average Hamming distance is determined as the similarity threshold.

[0067] Step S306: If the similarity between the adjacent frame detection images is greater than or equal to the similarity threshold, the detection image of the next frame in the adjacent frame detection images is determined as the target detection image of the object to be detected.

[0068] For example, if the similarity threshold is 5, the similarity between the first and second detected images is 2, and the similarity between the second and third detected images is 7, then it can be determined that the third detected image is significantly different from the second detected image, and the third detected image is identified as the target detection image of the object to be detected.

[0069] In some embodiments, before step S301, the process may further include: acquiring a video stream of the object to be detected; performing target recognition processing on the video stream of the object to be detected to obtain at least two detection images containing the object to be detected and the confidence level of each detection image; step S306 may include: if the similarity between adjacent detection images is greater than or equal to a similarity threshold, determining the detection image of the next frame in the adjacent detection images as a candidate image of the object to be detected; if there are at least two candidate images, obtaining the candidate image with the highest confidence level from all candidate images; and determining the candidate image with the highest confidence level as the target detection image of the object to be detected. Thus, by combining the confidence level of the detection image and the similarity between adjacent detection images, it helps to more accurately determine the target detection image of the object to be detected.

[0070] In this embodiment of the disclosure, a representative target detection image of the object to be detected can be quickly and accurately determined by using a determined similarity threshold.

[0071] In some embodiments, after step S305 is implemented, the following steps S311 to S312 may also be included:

[0072] Step S311: If the similarity between the adjacent frame detection images is less than the similarity threshold, determine a first ratio between the length and width of the detection image of the previous frame in the adjacent frame detection images and a second ratio between the length and width of the detection image of the next frame in the adjacent images; or, determine the number of frames between the detection image of the next frame in the adjacent frame detection images and the target detection image of the previous frame.

[0073] Here, we will take the example where the detection image is determined by the detection box corresponding to the video stream of the object to be detected. Since the size of the detection box of the detection image may be different, the length and width of each detection image may be different. During the movement of a pedestrian, if the pedestrian's behavior changes little, the size of the detection box tends to remain unchanged. However, if the pedestrian's behavior changes significantly, the size of the detection box will also change significantly. Therefore, the target detection image of the object to be detected can be determined based on the degree of change in the ratio between the length and width of the detection image. For example, the first ratio between the length and width of the first frame detection image is 0.8, and the second ratio between the length and width of the second frame detection image is 1.7.

[0074] If the behaviors of the objects to be detected in the video stream are too similar, resulting in large similarity and small differences between adjacent detection images, a hard sampling upper limit can be set to reduce the possibility of missed sampling in some cases when the number of collected images is too small. Here, the frame number of the target detection image in all detection images of the object to be detected can be counted. For example, the first frame of the target detection image is the first frame of all detection images of the object to be detected, and the second frame of the target detection image is the seventh frame of all detection images of the object to be detected.

[0075] Step S312: If the difference between the first ratio and the second ratio is greater than the difference threshold or the number of interval frames is greater than the number of frames threshold, the detection image of the next frame in the adjacent frame detection images is determined as the target detection image of the object to be detected.

[0076] For example: if the difference threshold is preset to 0.8, the first ratio between the length and width of the first frame detection image is 0.8, and the second ratio between the length and width of the second frame detection image is 1.7, then it is determined that the size difference of the detection box between the first frame detection image and the second frame detection image is large, and the second frame detection image can be identified as the target detection image; if the frame number threshold is preset to 10, and the second frame target detection image is the seventh frame of all detection images of the object to be detected, and the similarity of the detection images from the eighth to the seventeenth frame is less than the similarity threshold, then the seventeenth frame detection image can be identified as the target detection image.

[0077] In this embodiment of the present disclosure, when the similarity between adjacent frame detection images is less than a similarity threshold, a first ratio of the detection image of the previous frame in the adjacent frame detection images and a second ratio of the detection image of the subsequent frame in the adjacent images are determined; or, the number of interval frames between the detection image of the subsequent frame in the adjacent frame detection images and the target detection image of the previous frame are determined; then, based on the difference between the first ratio and the second ratio or the number of interval frames, the target detection image of the object to be detected is quickly and accurately determined, reducing situations such as missed sampling in some cases when the number of collected images is too small.

[0078] After filtering the detection images, if the target detection image includes at least two frames, the at least two target detection images can be fused to obtain the fused features of the object to be detected. This disclosure provides an image fusion method, such as... Figure 4 As shown, the method includes the following steps S401 to S405:

[0079] Step S401: Extract features from each of the target detection images to obtain the image features of each target detection image.

[0080] In the pedestrian re-identification database construction and retrieval process, related technologies involve selecting frames and extracting features from images captured by a camera, creating a pedestrian feature database based on the extracted features of each image, then selecting a specific pedestrian image from the captured images, extracting its features, and using these features to search the entire pedestrian feature database to find the person. However, building a database for each image separately consumes significant retrieval and database construction resources, resulting in low accuracy and poor performance. Alternatively, using a feature averaging method to fuse the overall features of each tracked person before database construction can lead to a significant bias in the fused features towards areas with higher sampling density, especially if the sampled images are taken from a particular angle (e.g., behind the person), resulting in inaccurate retrieval results. After filtering the detection images, if the target detection image includes at least two frames, these two frames can be fused to obtain the fused features of the target object.

[0081] Image features can be understood as the characteristics or properties that distinguish one type of image from other types of images. Each image possesses unique features that differentiate it from other images. For example, image features may include one or more of the following: brightness, edge, shape, texture, color, information content, and object type. Image features can be characterized using methods such as feature matrices, which helps improve processing efficiency. During step S401, one or more algorithms, such as Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP), or Difference of Gaussian (DOG), can be used to extract features from each target detection image, obtaining the image features of each target detection image.

[0082] Step S402: Based on the sequential order of each target detection image, determine at least two adjacent images for each target detection image.

[0083] Here, the sequential order of the target detection images can refer to the sequential order of their corresponding frame numbers. At least two adjacent images for each target detection image can refer to at least two target detection images whose frame numbers follow the current target detection image's frame number. For example, the three adjacent images of each target detection image can be determined. Since all detection images of the object to be detected have different acquisition orders (e.g., five frames of the object to be detected are obtained, with the first frame acquired earlier than the second frame, and the second frame acquired earlier than the third frame), the sequential order between target detection images can be determined based on the detection images. For example, if the first target detection image corresponds to the first frame, the second to the eighth frame, and the third to the fifteenth frame, then the frame number of the first target detection image comes first, the frame number of the second frame is in the middle, and the frame number of the third frame comes last. The second and third target detection images are two adjacent images of the first target detection image.

[0084] If a target detection image has only one neighboring image or no neighboring images among all target detection images, then at least two target detection images with frame numbers preceding the frame number of the current target detection image can be identified as neighboring images of the current target detection image. In other words, it is not necessary to obtain at least two target detection images with frame numbers following the frame number of the current target detection image. This is not a limitation.

[0085] Step S403: Based on at least two adjacent images of each target detection image, determine the connection relationship between each target detection image.

[0086] Here, the connection relationship between each target detection image can include: the second frame target detection image and the third frame target detection image are two adjacent images of the first frame target detection image, the third frame target detection image and the fourth frame target detection image are two adjacent images of the second frame target detection image, etc.

[0087] Step S404: Based on the image features and the connectivity of each target detection image, perform graph modeling on all target detection images to obtain the graph model corresponding to all target detection images.

[0088] Here, graph modeling can be understood as constructing the relationships between image features of a target detection image. Through graph modeling, a graph model can be obtained, which refers to a structure used to represent the relationships between image features of a target detection image. A graph model consists of nodes and edges. Nodes in the graph model can be image features of the target detection image, and edges can represent the connections between these image features. For example, a graph model may include a first node, a second node, a third node, a fourth node, etc. The first node is connected to the second and third nodes, and the second node is connected to the third and fourth nodes, etc. The first node represents the image features of the first target detection image, and the second node represents the image features of the second target detection image, etc.

[0089] Step S405: Based on the graph model corresponding to all the target detection images, determine the fusion features of the object to be detected.

[0090] Here, the image features of the first target detection image and other target detection images connected to the first target detection image can be fused in the first feature to obtain the first feature; the image features of the second target detection image and other target detection images connected to the second target detection image can be fused in the first feature to obtain the second feature, and so on; the first feature, the second feature, and other features can be fused in the second feature to obtain the fused feature of the object to be detected. For example, a preset first convolutional network can be used for the first feature fusion, and a preset second convolutional network can be used for the second feature fusion, etc. The fused feature of the object to be detected is used to perform re-identification processing on the object to be detected, such as storing the fused feature of the object to be detected in a pedestrian feature library, obtaining the detection image of the current pedestrian, extracting the image features of the detection image of the current pedestrian; determining the current similarity between the image features of the detection image of the current pedestrian and the fused feature of the object to be detected in the pedestrian feature library; if the current similarity meets the preset similarity condition, determining that the current pedestrian and the object to be detected are the same object; if the current similarity does not meet the preset similarity condition, determining that the current pedestrian and the object to be detected are different objects, etc.

[0091] In this embodiment of the disclosure, graph modeling is performed on all target detection images to obtain the graph model corresponding to all target detection images. Based on the graph model, the fusion features of the target object can be determined more accurately and quickly, so that the fusion features of the target object are richer and more accurate.

[0092] In some embodiments, step S405 may include the following steps S4051 to S4053:

[0093] Step S4051: Use the graph attention network of the target fusion model to extract features from the graph model to obtain initial features.

[0094] Here, a trained target fusion model can be used to fuse the graph model to obtain the fused features of the object to be detected. The target fusion model can refer to a pre-set machine learning model, such as a neural network model used to perform feature fusion. The graph model corresponding to the object detection image is input into the trained target fusion model to obtain the fused features of the object to be detected. The target fusion model can at least include Graph Attention Networks (GATs). Graph Attention Networks are a novel neural network architecture based on graph structure data. They utilize hidden self-attention layers to address the shortcomings of previous methods based on graph convolution or its approximations. Through stacked layers, nodes can participate in the features of their neighbors, implicitly assigning different weights to different nodes in the neighborhood without requiring any costly matrix operations (such as inversion) or prior knowledge of the graph structure. For example, the graph model corresponding to the object detection image is input into the graph attention network in the target fusion model for initial feature fusion to obtain initial features.

[0095] Step S4052: The initial features are fused using the convolutional network of the target fusion model to obtain the fused initial features.

[0096] Here, the target fusion model may also include at least a convolutional network (GraphPooling) for pooling multiple initial features to achieve further feature fusion. For example, the initial features can be input into the convolutional network of the target fusion model to obtain the fused initial features.

[0097] Step S4053: The initial features after fusion are mapped using the multilayer perceptron network of the target fusion model to obtain the fused features of the object to be detected.

[0098] Here, the target fusion model can also include at least a Multilayer Perceptron (MLP). A MLP is a feedforward artificial neural network model that maps multiple input datasets to a single output dataset, making the output features more closely resemble the final output result. For example, the fused initial features can be input into the MLP of the target fusion model to obtain the fused features of the object to be detected.

[0099] In this embodiment of the disclosure, by using networks with different functions in the target fusion model, the graph model can be fused multiple times quickly and accurately to obtain the fused features of the object to be detected.

[0100] In some embodiments, steps S411 to S414 may be included before step S4051:

[0101] Step S411: Obtain the sample graph model corresponding to the video stream containing the sample objects, as well as the standard image set.

[0102] Here, the sample object can refer to the object used to train the target fusion model, such as the first pedestrian being the object to be detected and the second pedestrian being the sample object. The nodes in the graph model are the image features of the video images in the video stream of the sample object; the sample object and the object to be detected are of the same type; the standard image set includes a first image subset and a second image subset. The standard images in the first image subset contain objects of the same type as the sample object, and the standard images in the second image subset contain objects of different types than the sample object; the differences represented by the different types can include: viewpoint differences, target deformation, lighting differences, scale differences, partial occlusion, complex backgrounds, and different shapes within the same type, etc., which are not limited here.

[0103] Step S412: Randomly select any standard image from the first image subset and the second image subset.

[0104] Here, during the training of the target fusion model, any type of standard image can be randomly selected from the standard image set to determine the loss of the target fusion model.

[0105] Step S413: Extract features from any of the standard images to obtain the image features of the standard images in the standard image set.

[0106] Here, one or more algorithms such as histogram of directional gradients, local binary mode, or Gaussian difference can be used to extract features from any standard image and obtain the image features of the standard image.

[0107] Step S414: Based on the image features of the sample image model and the standard image, train the initial fusion model to obtain the target fusion model.

[0108] Here, the untrained target fusion model (initial fusion model) can be used to perform feature fusion on the sample image model to obtain the fusion features of the sample object; the feature distance between the fusion features of the sample object and the image features of the standard image is determined, and this feature distance is determined as the current loss of the untrained target fusion model; the parameters of the untrained target fusion model are adjusted based on the current loss, and if the fusion accuracy of the untrained target fusion model meets the preset accuracy condition, the untrained target fusion model with adjusted parameters is determined as the trained target fusion model.

[0109] In this embodiment of the disclosure, the untrained target fusion model can be trained quickly and accurately using the sample graph model corresponding to the video stream of the sample object and the standard image set to obtain the trained target fusion model.

[0110] In some embodiments, steps S4141 to S4144 may be included before step S414 is implemented:

[0111] Step S4141: Based on the sample graph model, perform feature fusion on the image features of the video image to obtain the fused image features.

[0112] Here, the sample image model can be input into the untrained target fusion model to obtain the fused image features corresponding to the image features of the video image.

[0113] Step S4142: Determine the feature distance between the fused image features and the image features of the standard image.

[0114] Here, feature distance can refer to Euclidean distance, Manhattan distance, or cosine similarity between features, etc., and is not limited to these terms.

[0115] Step S4143: Based on the feature distance, determine the contrast loss of the initial fusion model.

[0116] Here, this feature distance can be determined as the contrastive loss of the initial fusion model.

[0117] Step S4144: Based on the contrast loss and target step size, adjust the parameters of the initial fusion model to obtain the target fusion model.

[0118] Here, the adjustment amount can be determined based on the contrastive loss and the target step size. The adjusted parameters are obtained by subtracting the parameters of the initial fusion model from the adjustment amount. If the fusion accuracy of the initial fusion model meets the preset accuracy condition, the adjusted parameters are then used as the parameters of the target fusion model. The target step size can be determined by the preset initial step size and the current number of updates.

[0119] In some embodiments, step S4143 may include: when the feature distance falls within a first numerical range, using a first loss function matching the first numerical range to determine a first loss matching the feature distance; when the feature distance falls within a second numerical range, using a second loss function matching the second numerical range to determine a second loss matching the feature distance; and determining a contrastive loss based on the first loss and the second loss. Here, by randomly sampling from other standard image sets of the same and different classes, and using an improved contrastive loss to update the initial fusion model, the aim is to make the distances between features of the same class similar and the distances between features of different classes separate. The purpose of improving the contrastive loss is to mine difficult samples, remove some samples with excessive losses, prevent label errors caused by other reasons such as erroneous tracking chains in the sample image model, and reduce the loss propagation of simple samples, effectively reducing the impact of noisy samples on the overall initial fusion model training. For example, the contrastive loss can be determined using the following formula:

[0120]

[0121]

[0122]

[0123]

[0124] In formulas (1), (2), (3), and (4), L represents the total loss of the initial fusion model, N represents the number of standard images randomly obtained from the standard image set, L′ represents the loss determined by a single standard image frame, margin is the critical value commonly used in contrastive learning, y is the label indicating whether the features are of the same class, d represents the cosine distance between features, and L p Indicates the first loss, L n This indicates a second loss, etc.

[0125] In this embodiment of the disclosure, the feature distance is determined by comparing the fused image features with the image features of the standard image, and then the contrast loss of the initial fusion model is determined by the feature distance. This helps to quickly and accurately adjust the parameters of the initial fusion model based on the contrast loss and the preset step size to obtain the target fusion model.

[0126] The following describes the application of the image filtering method provided in this disclosure in a real-world scenario, using a scenario where the object to be detected is a pedestrian as an example. In related technologies, traditional perceptual hash algorithms (Phash) are used to filter images. For example, after performing DCT transformation on the acquired image, a transformation matrix is ​​obtained, and only 8*8 frequency information is selected for Hamming distance calculation. Figure 5 As shown, the actual person re-identification (ReID) image 501 is transformed by DCT to obtain the transformation matrix 502. The frequency domain information in the transformation matrix 502 is relatively scattered. If only the 8*8 size frequency information in the transformation matrix 502 is taken, a lot of important information will be missed, resulting in a large deviation when calculating the similarity of the images, which is not conducive to the frame selection operation of the person re-identification image.

[0127] This disclosure proposes a novel fast similarity calculation method. After performing DCT transformation, high-frequency region feature values ​​of other parts of the image are collected, and the maximum value of the high-frequency region of the entire image is sampled. Finally, the Hamming distance is calculated by setting the weight of the sub-Hamming distance according to the frequency domain responsivity. This helps to consider more frequency information and is more accurate in describing image similarity.

[0128] This disclosure provides an image filtering method, such as... Figure 6 As shown, the method may include the following steps S601 to S611:

[0129] Step S601: Obtain the video stream of the object to be detected.

[0130] Here, pre-set cameras and other equipment can be used to capture images of the object to be detected, thus obtaining a video stream of the object.

[0131] Step S602: Perform target detection processing on the video stream of the object to be detected to obtain the detection result.

[0132] Here, the video stream can be segmented to obtain multiple consecutive video frames; target detection processing is performed on each video frame to obtain detection results; the detection includes whether the target object is detected, and if the target object is detected, the position and size of the detection box corresponding to the target object, as well as the confidence score of each detection image; if the confidence score is greater than a preset confidence threshold, it is determined that the target object is detected; if the confidence score is less than or equal to the preset confidence threshold, it is determined that the target object is not detected, etc.

[0133] Step S603: Obtain at least two detection images containing the object to be detected from the detection results.

[0134] Here, the video image can be cropped based on the position and size of the detection box in the detection result to obtain at least two frames of detection images of the object to be detected.

[0135] Step S604: Determine the Hamming distance between adjacent frame detection images.

[0136] Here, a DCT transform can be performed on each detected image to obtain a transform matrix; frequency sets of different frequency bands are obtained from different positions of the transform matrix; based on the frequency sets of the same frequency band, sub-Hamming distances are determined; and a weighted sum of all sub-Hamming distances is performed to obtain the Hamming distance between adjacent frame detected images. After determining the Hamming distance between adjacent frame detected images, step S609 can be entered to store the Hamming distances in a preset list, so that step S605 can be entered to sort all the Hamming distances in the list to obtain a Hamming distance threshold; alternatively, step S606 can be entered to compare the Hamming distance with the threshold to determine whether the detected image of the next frame in the adjacent frame detected images is the target detected image. Figure 7 As shown, the frequency set of the first frequency band can be obtained from the upper left region 701 of the transformation matrix, the frequency set of the second frequency band can be obtained from the left side region 702 and the upper side region 703 of the transformation matrix, and the frequency set of the third frequency band can be obtained from the remaining region 704 of the transformation matrix.

[0137] Step S605: Determine the Hamming distance threshold based on all Hamming distances.

[0138] Here, the Hamming distances between all adjacent frame detection images can be sorted to obtain the sorting results; a preset number of Hamming distances (such as the top 10% of Hamming distances) can be obtained, the average value of the preset number of Hamming distances can be determined, and this average value can be determined as the Hamming distance threshold.

[0139] Step S606: Determine whether the Hamming distance is greater than the Hamming distance threshold.

[0140] Here, it can refer to the Hamming distance between adjacent frame detection images. If the Hamming distance between adjacent frame detection images is greater than the Hamming distance threshold, then proceed to step S609; if the Hamming distance between adjacent frame detection images is less than or equal to the Hamming distance threshold, then proceed to step S607.

[0141] Step S607: Determine whether the difference between the aspect ratio values ​​is greater than the difference threshold.

[0142] Here, it can refer to the difference between the first ratio between the length and width of the detection image of the previous frame in the adjacent frame detection image and the second ratio between the length and width of the detection image of the next frame in the adjacent image. If it is determined that the difference is greater than a preset difference threshold, then proceed to step S609; if it is determined that the difference is less than or equal to the preset difference threshold, then proceed to step S608.

[0143] Step S608: Determine whether the interval frame number is greater than the frame number threshold.

[0144] Here, it can refer to the number of frames between the detection image of the next frame and the target detection image of the previous frame in adjacent frame detection images. If the number of frames between the frames is greater than a preset frame number threshold, then proceed to step S609; if the number of frames between the frames is less than or equal to the preset frame number threshold, then proceed to step S604 to continue determining the Hamming distance between the next adjacent frame detection images.

[0145] Step S609: Label the detected image to obtain the labeling result.

[0146] Here, if the Hamming distance between adjacent detected images is greater than a Hamming distance threshold, or the difference in aspect ratios is greater than a difference threshold, or the number of frames between them is greater than a preset frame number threshold, then the detected image of the next frame in the adjacent detected images is marked to obtain candidate detected images. For example... Figure 8 As shown, by marking the detection images of the object to be detected, it can be determined that the second frame detection image 801, the seventh frame detection image 802, and other detection images can be candidate detection images.

[0147] Step S610: Obtain the confidence level of each detected image from the detection results.

[0148] For example, the confidence level of the first detected image is 0.75, and the confidence level of the second detected image is 0.8, etc.

[0149] Step S611: Based on the confidence level and labeling results, determine the target detection image of the object to be detected.

[0150] Here, if the confidence level of the first candidate detection image is 0.8, the confidence level of the second candidate detection image is 0.9, and the confidence level of the third candidate detection image is 0.95, then the third candidate detection image can be determined as the target detection image of the object to be detected.

[0151] In this embodiment of the disclosure, after filtering the detection images, if the target detection image includes at least two frames, the at least two target detection images can be fused to obtain the fused features of the object to be detected. For example... Figure 9As shown, if multiple frames of target detection images exist, these images can be fused without needing to save the image features of each individual frame. In this embodiment, a graph model and a target fusion model with a multilayer perceptron structure can be built to fuse the graph models corresponding to multiple frames of target detection images, resulting in fused features. The target fusion model is trained using a self-supervised learning method, which not only better learns the structural relationships between the image features of the target detection images but also performs dynamic feature fusion, making the fused features more stable. This leads to higher precision and recall in the overall person re-identification system and makes the fused features more closely resemble the individual image features used for retrieval during person re-identification.

[0152] For training the target fusion model, in this embodiment, the initial fusion model is trained by comparative learning using randomly collected standard images of the same and different classes, enabling the initial fusion model to better learn image features between the same and different classes. Since the image features used are both single-image features and fusion features corresponding to connection relationships or tracking IDs in the graph model, the fused features better match the single-image features. Simultaneously, image feature sampling and training of standard images can be performed without annotations, allowing learning and training without labeled data. Furthermore, this scheme also adaptively improves the loss function to prevent noisy samples from affecting the overall model training.

[0153] This disclosure proposes a learnable video pedestrian re-identification feature fusion scheme. First, the image features of the target detection image are processed using graph modeling to obtain a graph model, such as... Figure 10 As shown, the first frame target detection image 1001 is connected to the second frame target detection image 1002, the third frame target detection image 1003, and the fourth frame target detection image 1004; the second frame target detection image 1002 is connected to the third frame target detection image 1003, the fourth frame target detection image 1004, and the fifth frame target detection image 1005, etc.; then, the convolutional network and graph pooling layer in the target fusion model are used to learn and fuse the sequential structure and image features of the target detection images.

[0154] This allows the graphical model to more easily learn the image features of the later target detection images in the sequence when selecting feature weights. This facilitates skipping similar, repetitive target detection images and prevents target detection images from falling into local feature traps. Furthermore, by comparing the image features with those of randomly collected standard images, the parameters of the initial fusion model are updated, ultimately resulting in a more robust fusion feature set. Figure 11As shown, the target fusion model may include a two-layer graph attention network (first graph attention network 1101 and second graph attention network 1104), a two-layer convolutional network (first convolutional network 1102 and second convolutional network 1105), a multilayer perceptron network 1107, and readout functions (first readout function 1103 and second readout function 1106) for outputting intermediate features during processing, etc. Figure 12 As shown, during the training process, the contrast loss 1204 can be determined first based on the sample graph model and the standard image set 1201; the initial fusion model is trained using the contrast loss 1204 to obtain the target fusion model 1203; during use, the graph model 1202 corresponding to the target detection image of the object to be detected can be input into the target fusion model 1203 to obtain the fusion feature 1205, and the fusion feature 1205 is stored in the pedestrian feature library 1206 for re-identification processing of the object to be detected, etc.

[0155] Based on the foregoing embodiments, this disclosure provides an image screening device, which includes various units and modules included in each unit, and can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0156] Figure 13 This is a schematic diagram of the composition structure of an image filtering device provided in an embodiment of the present disclosure, as shown below. Figure 13 As shown, the image filtering device 1300 includes: a first acquisition module 1310, a first determination module 1320, a second acquisition module 1330, a second determination module 1340, and a third acquisition module 1350, wherein:

[0157] The first acquisition module 1310 is used to acquire at least two detection images containing an object to be detected; the first determination module 1320 is used to determine the frequency information of each detection image; wherein the frequency information is used to characterize the information change of pixels in the detection image; the second acquisition module 1330 is used to acquire the frequency set of at least two frequency bands corresponding to each detection image from the frequency information of each detection image; the second determination module 1340 is used to determine the similarity between adjacent detection images based on the frequency set of at least two frequency bands corresponding to each of the at least two detection images; the third acquisition module 1350 is used to acquire the target detection image of the object to be detected from the at least two detection images based on the similarity between the adjacent detection images.

[0158] In some embodiments, the second acquisition module is further configured to: acquire a frequency set of a first frequency band from the upper left region of a transformation matrix used to characterize the frequency information; wherein the transformation matrix is ​​obtained by performing a discrete cosine transform on the detected image; acquire a frequency set of a second frequency band from the left and upper regions of the transformation matrix; acquire a frequency set of a third frequency band from the remaining region of the transformation matrix; wherein the remaining region is the region in the transformation matrix other than the upper left region, the left region, and the upper region; the first frequency band is higher than the second frequency band, and the second frequency band is higher than the third frequency band.

[0159] In some embodiments, the second determining module is further configured to: binarize the frequency sets of at least two frequency bands corresponding to each of the adjacent frame detection images to obtain frequency sets after frequency band binarization; determine the sub-Hamming distance of each common frequency band of the adjacent frame detection images based on the frequency sets after frequency band binarization; and perform a weighted summation of the sub-Hamming distances of all the common frequency bands to obtain the Hamming distance between the adjacent frame detection images; wherein the Hamming distance between the adjacent frame detection images characterizes the similarity between the adjacent frame detection images.

[0160] In some embodiments, the third acquisition module is further configured to: determine a similarity threshold based on the Hamming distance between all adjacent frame detection images; and, if the similarity between the adjacent frame detection images is greater than or equal to the similarity threshold, determine the detection image of the next frame in the adjacent frame detection images as the target detection image of the object to be detected; wherein the third acquisition module is further configured to: acquire a specified number of first Hamming distances from all the Hamming distances; wherein the specified number of first Hamming distances is greater than or equal to other Hamming distances among all the Hamming distances except the specified number of first Hamming distances; and determine a distance threshold based on the specified number of first Hamming distances as the similarity threshold.

[0161] In some embodiments, the apparatus further includes: a third determining module, configured to: determine a first ratio between the length and width of the detection image of the previous frame in the adjacent frame detection images and a second ratio between the length and width of the detection image of the subsequent frame in the adjacent images, when the similarity between the adjacent frame detection images is less than the similarity threshold; or, determine the number of interval frames between the detection image of the subsequent frame in the adjacent frame detection images and the target detection image of the previous frame; and determine the detection image of the subsequent frame in the adjacent frame detection images as the target detection image of the object to be detected when the difference between the first ratio and the second ratio is greater than a difference threshold or the number of interval frames is greater than a frame number threshold.

[0162] In some embodiments, the target detection image includes at least two frames; the apparatus further includes: a first extraction module, configured to extract features from each target detection image to obtain image features of each target detection image; a fourth determination module, configured to determine at least two adjacent images of each target detection image based on the sequential order of each target detection image; a fifth determination module, configured to determine the connection relationship between each target detection image based on at least two adjacent images of each target detection image; a modeling module, configured to perform graph modeling on all target detection images based on the image features of each target detection image and the connection relationship to obtain a graph model corresponding to all target detection images; and a sixth determination module, configured to determine the fusion features of the object to be detected based on the graph model corresponding to all target detection images; wherein the fusion features of the object to be detected are used for re-identification processing of the object to be detected.

[0163] In some embodiments, the sixth determining module is further configured to: extract features from graph modeling using a graph attention network of the target fusion model to obtain initial features; fuse the initial features using a convolutional network of the target fusion model to obtain fused initial features; and map the fused initial features using a multilayer perceptron network of the target fusion model to obtain fused features of the object to be detected.

[0164] In some embodiments, the apparatus further includes: a fourth acquisition module, configured to acquire a sample graph model corresponding to a video stream containing sample objects, and a standard image set; wherein, the nodes in the graph model are image features of video images in the video stream of the sample objects; the sample objects and the object to be detected are of the same type; the standard image set includes a first image subset and a second image subset, wherein the standard images in the first image subset contain objects of the same type as the sample objects, and the standard images in the second image subset contain objects of different types than the sample objects; a fifth acquisition module, configured to randomly acquire any standard image from the first image subset and the second image subset; a second extraction module, configured to perform feature extraction on the any standard image to obtain image features of the standard images in the standard image set; and a training module, configured to train an initial fusion model based on the sample graph model and the image features of the standard images to obtain the target fusion model.

[0165] In some embodiments, the training module is further configured to: perform feature fusion on the image features of the video image based on the sample image model to obtain fused image features; determine the feature distance between the fused image features and the image features of the standard image; determine the contrast loss of the initial fusion model based on the feature distance; and adjust the parameters of the initial fusion model based on the contrast loss and the target step size to obtain the target fusion model.

[0166] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0167] It should be noted that, in the embodiments of this disclosure, if the above-described image screening method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0168] This disclosure provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0169] This disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0170] This disclosure provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0171] This disclosure provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0172] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0173] It should be noted that, Figure 14 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this disclosure, such as... Figure 14 As shown, the hardware entity of the computer device 1400 includes: a processor 1401, a communication interface 1402, and a memory 1403, wherein:

[0174] Processor 1401 typically controls the overall operation of computer device 1400.

[0175] Communication interface 1402 enables computer devices to communicate with other terminals or servers via a network.

[0176] The memory 1403 is configured to store instructions and applications executable by the processor 1401, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1401 and various modules in the computer device 1400. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 1401, the communication interface 1402, and the memory 1403 can be performed via bus 1404.

[0177] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0178] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0179] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0180] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0181] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0182] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0183] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0184] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0185] If the embodiments of this disclosure involve personal information, the products using these embodiments have clearly informed the users of the personal information processing rules and obtained their voluntary consent before processing the personal information. If the embodiments of this disclosure involve sensitive personal information, the products using these embodiments have obtained the individual's separate consent before processing the sensitive personal information, and the requirement of "express consent" is also met.

[0186] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image filtering method, characterized in that, include: Acquire at least two detection images containing the object to be detected; Determine the frequency information of each of the detected images; wherein the frequency information is used to characterize the information changes of pixels in the detected images; Obtain a frequency set of at least two frequency bands corresponding to each of the detected images from the frequency information of each detected image; The similarity between adjacent detection images is determined based on the frequency sets of at least two frequency bands corresponding to each of the at least two detection images; wherein the similarity between adjacent detection images is calculated based on the Hamming distance of each same frequency band. Based on the similarity between the adjacent frame detection images, the target detection image of the object to be detected is obtained from the at least two frame detection images.

2. The method according to claim 1, characterized in that, The step of obtaining a frequency set of at least two frequency bands corresponding to each of the detected images from the frequency information of each detected image includes: The frequency set of the first frequency band is obtained from the upper left region of the transformation matrix used to characterize the frequency information; wherein, the transformation matrix is ​​obtained by performing a discrete cosine transform on the detected image; The frequency set of the second frequency band is obtained from the left and upper regions of the transformation matrix; The frequency set of the third frequency band is obtained from the remaining region of the transformation matrix; wherein, the remaining region is the region in the transformation matrix excluding the upper left corner region, the left side region and the upper side region; the first frequency band is higher than the second frequency band, and the second frequency band is higher than the third frequency band.

3. The method according to claim 1, characterized in that, Determining the similarity between adjacent detection images based on the frequency sets of at least two frequency bands corresponding to each of the at least two detected images includes: Binarize the frequency sets of at least two frequency bands corresponding to each of the adjacent frame detection images to obtain the frequency sets after frequency band binarization. Based on the frequency set after the frequency band is binarized, the sub-Hamming distance of each same frequency band in the adjacent frame detection images is determined; The Hamming distances of all the same frequency bands are weighted and summed to obtain the Hamming distance between the adjacent frame detection images; wherein, the Hamming distance between the adjacent frame detection images represents the similarity between the adjacent frame detection images.

4. The method according to claim 3, characterized in that, The step of obtaining the target detection image of the object to be detected from the at least two detection images based on the similarity between the adjacent detection images includes: The similarity threshold is determined based on the Hamming distance between all adjacent frame detection images; If the similarity between adjacent frame detection images is greater than or equal to the similarity threshold, the detection image of the next frame in the adjacent frame detection images is determined as the target detection image of the object to be detected. The step of determining the similarity threshold based on the Hamming distance between all adjacent frame detection images includes: Obtain a specified number of first Hamming distances from all the Hamming distances; wherein the specified number of first Hamming distances is greater than or equal to all other Hamming distances except the specified number of first Hamming distances; determine a distance threshold based on the specified number of first Hamming distances as the similarity threshold.

5. The method according to claim 4, characterized in that, The method further includes: If the similarity between the adjacent frame detection images is less than the similarity threshold, a first ratio between the length and width of the detection image of the previous frame in the adjacent frame detection images and a second ratio between the length and width of the detection image of the next frame in the adjacent frame detection images are determined; or, the number of frames between the detection image of the next frame in the adjacent frame detection images and the target detection image of the previous frame are determined. If the difference between the first ratio and the second ratio is greater than the difference threshold or the number of frames between the intervals is greater than the number of frames threshold, the detection image of the next frame in the adjacent frame detection images is determined as the target detection image of the object to be detected.

6. The method according to any one of claims 1 to 5, characterized in that, The target detection image comprises at least two frames; the method further includes: Feature extraction is performed on each of the target detection images to obtain the image features of each target detection image; Based on the sequential order of each target detection image, at least two adjacent images of each target detection image are determined; Based on at least two adjacent images of each of the target detection images, the connection relationship between the target detection images is determined; Based on the image features and the connection relationships of each target detection image, graph modeling is performed on all target detection images to obtain the graph model corresponding to all target detection images; Based on the graph model corresponding to all the target detection images, the fusion features of the object to be detected are determined; wherein, the fusion features of the object to be detected are used to perform re-identification processing on the object to be detected.

7. The method according to claim 6, characterized in that, The step of determining the fusion features of the object to be detected based on the graph model corresponding to all the target detection images includes: The graph attention network of the target fusion model is used to extract features from the graph model to obtain initial features; The initial features are fused using the convolutional network of the target fusion model to obtain the fused initial features; The initial features after fusion are mapped using the multilayer perceptron network of the target fusion model to obtain the fused features of the object to be detected.

8. The method according to claim 7, characterized in that, The method further includes: Obtain a sample graph model corresponding to a video stream containing sample objects, and a standard image set; wherein, the nodes in the graph model are image features of video images in the video stream of the sample objects; the sample objects and the objects to be detected are of the same type; the standard image set includes a first image subset and a second image subset, wherein the standard images in the first image subset contain objects of the same type as the sample objects, and the standard images in the second image subset contain objects of different types than the sample objects; Randomly select any standard image from the first image subset and the second image subset; Feature extraction is performed on any of the standard images to obtain the image features of the standard images in the standard image set; Based on the image features of the sample image model and the standard image, the initial fusion model is trained to obtain the target fusion model.

9. The method according to claim 8, characterized in that, The process of training the initial fusion model based on the image features of the sample image model and the standard image to obtain the target fusion model includes: Based on the sample graph model, the image features of the video image are fused to obtain the fused image features; Determine the feature distance between the fused image features and the image features of the standard image; Based on the feature distance, the contrast loss of the initial fusion model is determined; Based on the contrast loss and the target step size, the parameters of the initial fusion model are adjusted to obtain the target fusion model.

10. An image filtering device, characterized in that, include: The first acquisition module is used to acquire at least two frames of detection images containing the object to be detected; The first determining module is used to determine the frequency information of each of the detected images; wherein the frequency information is used to characterize the information changes of pixels in the detected images; The second acquisition module is used to acquire a frequency set of at least two frequency bands corresponding to each detection image from the frequency information of each detection image; The second determining module is used to determine the similarity between adjacent detection images based on the frequency sets of at least two frequency bands corresponding to each of the at least two detection images; wherein the similarity between adjacent detection images is generated based on the Hamming distance of each same frequency band. The third acquisition module is used to acquire the target detection image of the object to be detected from the at least two detection images based on the similarity between the adjacent frame detection images.

Images