Image retrieval methods, apparatus, computer equipment and storage media

Abstract

This invention discloses an image retrieval method, apparatus, computer device, and storage medium, relating to the field of image processing technology. The image retrieval method of this invention includes: acquiring an image of a target person corresponding to a target person and an image captured by a camera within a preset area; determining the image similarity between the target person image and the captured image, and dividing the captured image into a first target image and an image to be corrected based on the image similarity and a preset similarity threshold; acquiring a device association matrix corresponding to the camera within the preset area, and correcting the image similarity of the image to be corrected based on the device association matrix to obtain a corrected similarity for the image to be corrected; and selecting a second target image from the images to be corrected based on the corrected similarity. This invention improves the recall rate of captured images and the accuracy in scenarios involving image-based monitoring or analysis.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image retrieval method, apparatus, computer device, and storage medium. Background Technology

[0002] In the field of video surveillance, cameras are often used to capture facial photos, and the similarity between the captured facial images and pre-recorded facial images in a database is calculated to monitor people.

[0003] In existing technologies, a similarity threshold is typically set to determine whether a captured facial image contains the face of the target person. However, when the shooting equipment is of poor performance or environmental factors affect the image, many captured images containing the target person's face may not be recalled, resulting in a low image recall rate. This significantly reduces the number of recalled images, leading to lower accuracy in video surveillance scenarios. Summary of the Invention

[0004] This invention provides an image retrieval method, apparatus, computer device, and storage medium to address the problem of low accuracy in video surveillance scenarios due to the limited number of images retrieved in the prior art.

[0005] An image retrieval method includes:

[0006] Acquire the target person's image and the captured image obtained by a camera within a preset area; one camera corresponds to at least one captured image;

[0007] Determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, divide the captured image into a first target image and an image to be corrected;

[0008] Obtain the device association matrix corresponding to the shooting devices in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity corresponding to the image to be corrected;

[0009] Based on the corrected similarity, a second target image is selected from the images to be corrected, and the first target image and the second target image are used as the recall images corresponding to the target person.

[0010] An image recall device, comprising:

[0011] The image acquisition module is used to acquire the target person image corresponding to the target person and the captured image obtained by the shooting device within the preset area; one shooting device corresponds to at least one captured image;

[0012] An image segmentation module is used to determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, to segment the captured image into a first target image and an image to be corrected;

[0013] The similarity correction module is used to obtain the device association matrix corresponding to the shooting devices in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity.

[0014] The recall image determination module is used to filter out a second target image from the images to be corrected based on the corrected similarity, and use the first target image and the second target image as the recall images corresponding to the target person.

[0015] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image retrieval method described above.

[0016] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image retrieval method.

[0017] The aforementioned image retrieval method, apparatus, computer equipment, and storage medium, through which the method divides captured images using a preset similarity threshold, can initially separate first target images that clearly identify the target person's face from images that cannot be determined to be the target person's face, reducing the number of images to be corrected and improving the efficiency of image similarity correction. By introducing a device association matrix, relying on the shooting associations between shooting devices, the image similarity of the images to be corrected is adjusted, thereby making the image similarity of the images to be corrected that are more likely to contain the target person's face approach the preset similarity threshold. Then, based on the corrected similarity, second target images that may also be the target person's face are filtered from the images to be corrected. Thus, the recall rate of captured images in the image shooting scenario is improved, thereby improving the accuracy in scenarios based on image monitoring or analysis. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a schematic diagram of an application environment for an image retrieval method according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of an image retrieval method according to an embodiment of the present invention;

[0021] Figure 3 This is a principle block diagram of an image retrieval device according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The image retrieval method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment is shown. Specifically, this image retrieval method is applied in an image retrieval system, which includes, for example, […]. Figure 1 The diagram illustrates a client and server that communicate over a network to address the issue of low accuracy in video surveillance scenarios due to the limited number of images retrieved in existing technologies. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0025] In one embodiment, such as Figure 2 As shown, an image retrieval method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0026] S10: Acquire the target person image corresponding to the target person and the captured image obtained by the shooting device in the preset area; one shooting device corresponds to at least one captured image.

[0027] Understandably, the target person can be set according to different scenarios. For example, in a community monitoring scenario, the target person could be a resident of the community; in a shell company monitoring scenario, the target person could be an employee of the company. The target person image is an image containing the target person's face. The preset area can also be set according to different scenarios. For example, in a community monitoring scenario, the preset area could be the entire community. In a shell company monitoring scenario, the preset area could be the area where the company's building is located. The captured image is an image captured by a camera or other capturing device within the preset area. To facilitate determining the image similarity between the target person image and the captured image in step S20, the images captured by the capturing device can be filtered, that is, images containing at least one face can be selected as captured images, thereby improving the efficiency of image comparison. Within a time period, the capturing device can capture multiple images, and the final selected captured image containing at least one face may be one or more.

[0028] S20: Determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, divide the captured image into a first target image and an image to be corrected.

[0029] Understandably, image similarity represents the degree of similarity between an image of a target person and a captured image. This image similarity can be determined using image comparison algorithms, and it is primarily used to determine whether the person in the captured image and the person in the target person image are the same person. In this application, image similarity mainly refers to the feature similarity between facial features extracted from the face in the captured image and facial features extracted from the face in the target person image. A preset similarity threshold can be set according to different scenario requirements. For example, the preset similarity threshold can be set to 0.9 or 0.95, etc. When the image similarity exceeds the preset similarity threshold, the face in the captured image can be considered the face of the target person. Therefore, the captured image can be divided into a first target image and an image to be corrected using the preset similarity threshold and the image similarity. The first target image refers to the captured image with an image similarity greater than or equal to the preset similarity threshold. The image to be corrected refers to the captured image with an image similarity less than the preset similarity threshold.

[0030] Specifically, after determining the image similarity between the target person image and the captured image using an image comparison algorithm (such as comparing the grayscale values ​​of each frame between the target person image and the captured image), the image similarity can be compared with a preset similarity threshold. The captured image with an image similarity greater than or equal to the preset similarity threshold is then determined as the first target image. The captured image with an image similarity less than the preset similarity threshold is determined as the image to be corrected.

[0031] S30: Obtain the device association matrix corresponding to the shooting device in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity of the image to be corrected.

[0032] Understandably, the device association matrix represents the association between shooting devices within a preset area, and includes N rows and N columns of parameters. The Nth row represents the statistics of the number of times a person is captured by other shooting devices within the device association time t after being captured by the shooting device n. The Nth column represents the statistics of the number of times a person is captured by other shooting devices within the device association time t before being captured by the shooting device n. For example, assume that there is a parameter r in the device association matrix. ui This parameter represents the position of the person in the shooting device c. u The captured device c is associated with the device within the time t after the snapshot. i The statistical data captured. Among them, the device association time can be determined based on the density of the shooting devices distributed in the preset area and the capture rate. The density can be determined based on the heat map corresponding to the preset area, and the capture rate can generally be determined based on the average total time taken by different shooting devices to capture the same person five times in a row.

[0033] Furthermore, existing image retrieval methods only recall images with a similarity threshold greater than or equal to a preset threshold, resulting in a low recall rate and impacting the accuracy of personnel monitoring. However, images with similarity thresholds may also contain individuals. For example, if a person is captured by one camera, they may be captured by another camera, but due to lower performance, the similarity of the captured image may be less than the preset threshold. Therefore, this embodiment introduces a device association matrix between cameras to correct the image similarity of the image to be corrected. This corrected similarity approximates the original image similarity, allowing for the recall of a subset of images based on the corrected similarity, thus improving the overall image recall rate.

[0034] S40: Based on the corrected similarity, select the second target image from the images to be corrected, and use the first target image and the second target image as the recall images corresponding to the target personnel.

[0035] Specifically, after determining the correction similarity of the images to be corrected, the images are sorted in descending order of correction similarity. A preset number of images (the preset number can be set according to needs, for example, 5) are then selected as the second target images. In this way, all the first and second target images can be used as recall images corresponding to the target personnel. Based on these recall images, the target personnel can be analyzed (e.g., analysis of movement trajectories or frequency of entry and exit from locations).

[0036] In this embodiment, by dividing the captured images using a preset similarity threshold, a first target image clearly identifying the target person's face and images that cannot be determined to be the target person's face can be initially separated, reducing the number of images to be corrected and improving the efficiency of image similarity correction. By introducing a device association matrix, relying on the shooting associations between shooting devices, the image similarity of the images to be corrected is adjusted, so that the image similarity of the images to be corrected that are more likely to contain the target person's face approaches the preset similarity threshold. Then, based on the corrected similarity, a second target image that may also be the target person's face is selected from the images to be corrected. In this way, the recall rate of captured images in the image shooting scenario is improved, thereby improving the accuracy in scenarios where images are used for monitoring or analysis.

[0037] In one embodiment, step S30, namely obtaining the device association matrix corresponding to the shooting device, includes:

[0038] (1) Obtain the comparison image of the sample personnel and the sample image captured by the shooting device in the preset area; one shooting device corresponds to at least one sample image; one sample image corresponds to one sample shooting time.

[0039] Understandably, the sample personnel can be set according to different scenarios. For example, in a community monitoring scenario, the sample personnel could be a resident of the community; in a shell company monitoring scenario, the sample personnel could be an employee of the company. The preset area is the same as the preset area defined in step S10 above. The sample image is the image captured by the camera in the preset area, and the sample image can also be an image containing at least one face that has been pre-screened. The sample shooting time is the time when the camera captures the sample image.

[0040] (2) Determine the sample similarity between the comparison personnel image and the sample image, and determine all sample images with a sample similarity greater than the preset similarity threshold as sample matching images to obtain multiple sample matching images.

[0041] Specifically, after acquiring the images of the personnel and the sample images, an image comparison algorithm can be used to determine the sample similarity between the images of the personnel and the sample images. This sample similarity characterizes the degree of similarity between the images of the personnel and the sample images. The sample similarity is compared with a preset similarity threshold, and the sample images with a sample similarity greater than the preset similarity threshold are determined as sample matching images, thereby obtaining multiple sample matching images.

[0042] (3) Based on the shooting time of each sample matching image and the shooting device information to which each sample matching image belongs, determine the device correlation coefficient between any two shooting devices in the preset area.

[0043] Understandably, the imaging device information can be the device ID of the imaging device to which the sample image belongs. After dividing the preset area, each imaging device within that area can be assigned a unique ID, thus distinguishing images captured by different imaging devices within the preset area. Furthermore, the device correlation coefficient characterizes the degree of correlation between two imaging devices within the preset area. The device correlation coefficient may differ between any two different imaging devices.

[0044] (4) Based on the device correlation coefficient between any two shooting devices in the preset area, determine the device correlation matrix corresponding to the shooting devices in the preset area.

[0045] Understandably, the device correlation coefficient represents the statistical quantity of a person captured by one camera device being captured by another camera device within a preset device correlation time t after the first camera device captures the person. For example, assume there is a parameter r in the device correlation matrix. ui This parameter represents the position of the person in the shooting device c. u The captured device c is associated with the device within the time t after the snapshot. i The captured statistics. And consequently, the parameter r. ui The specific value refers to the correlation coefficient between the image and its corresponding device. Furthermore, before determining the device correlation coefficient based on the shooting time and the shooting device information of each sample matching image, the initial value of the device correlation coefficient between any two shooting devices can be set to 0. Therefore, after determining the device correlation coefficient between any two shooting devices within the preset area, the corresponding coefficient positions in the matrix can be filled according to the device correlation coefficient between any two shooting devices within the preset area, thereby obtaining the device correlation matrix corresponding to the shooting devices within the preset area.

[0046] In one embodiment, the device correlation coefficient between any two shooting devices within a preset area is determined based on the shooting time of each sample matching image and the shooting device information to which each sample matching image belongs, including the following steps:

[0047] (1) Compare the time difference between the shooting times of any two sample matching images, and determine the two sample matching images with a time difference less than the preset device association time as the associated image group to obtain multiple associated image groups.

[0048] Understandably, the preset device association time can be determined based on the density and capture rate of the shooting devices distributed in the preset area. The density can be determined based on the heat map corresponding to the preset area, and the capture rate can generally be determined based on the average total time taken by different shooting devices to capture the same person five times consecutively.

[0049] Specifically, after selecting matching images from the sample images based on sample similarity, the difference between the shooting times of any two matching images can be determined as the time difference between these two matching images. A preset device association time is then obtained, and this preset device association time and time difference are compared. Two matching images with a time difference less than the preset device association time are identified as a group of associated images. By iterating through all matching images, multiple groups of associated images can be obtained. Thus, a group of associated images represents the same person appearing in two different matching images within a preset device association time; that is, the person was photographed by the camera corresponding to one matching image and then by the camera corresponding to another matching image within the same device association time.

[0050] For example, in an image group, one sample matching image was captured at 3:50 PM on September 10, 2022, and another sample matching image was captured at 3:55 PM on September 10, 2022. The time difference between these two sample matching images is five minutes. Assuming the preset device association time is set to 10 minutes, these two sample matching images are combined into an associated image group.

[0051] (2) For each associated image group, the shooting device to which the sample matching image with the earlier shooting time belongs in the associated image group is determined as the earlier shooting device, and the shooting device to which the sample matching image with the later shooting time belongs in the associated image group is determined as the later shooting device.

[0052] Specifically, after identifying two matched sample images with a time difference less than a preset device association time as an associated image group, and obtaining multiple associated image groups, within each associated image group, the devices are categorized as "earlier" and "later" based on the shooting time of the two matched sample images. That is, the shooting device to which the matched sample image with the earlier shooting time belongs is determined as the "earlier" device, and the shooting device to which the matched sample image with the later shooting time belongs is determined as the "later" device. It should be noted that the "earlier" and "later" devices may be the same shooting device (when both sample images belong to the same shooting device) or different shooting devices (when the two sample images do not belong to the same shooting device).

[0053] (3) Based on the shooting device information of the two sample matching images in the associated image group, classify the multiple associated image groups to obtain the associated image groups corresponding to the same pair of front and back devices.

[0054] Specifically, for each associated image group, after determining the capturing device of the sample matching image captured earlier in the associated image group as the "preceding device" and the capturing device of the sample matching image captured later in the associated image group as the "following device," multiple associated image groups can be classified based on the capturing device information of the two sample matching images in the associated image group. This allows for the identification of associated image groups corresponding to the same pair of preceding and following devices. For example, suppose the preceding device of an associated image group is c. u The device in the latter is c i Then, among all associated image groups, the one with the front device being c... u The device in the latter is c i The associated image groups are grouped into one category.

[0055] (4) Based on the number of associated image groups corresponding to each pair of front and rear devices, determine the device association coefficient between any two shooting devices within the preset area.

[0056] Specifically, after classifying multiple associated image groups based on the imaging device information of the two sample matching images in the associated image group, and obtaining the associated image groups corresponding to the same pair of foreground and background devices, the number of associated image groups corresponding to each different foreground and background device can be determined. Thus, the device association coefficient between any two imaging devices within a preset area is determined based on the number of associated image groups in each different category.

[0057] For example, suppose there is a previous device c u The device in the latter is c i If there are five associated image groups, then their corresponding parameter r in the device association matrix is... uiThe device correlation coefficient is 5. Assume there exists a preceding device c. d The device in the latter is c i If there are ten associated image groups, then their corresponding parameter r in the device association matrix is... di The equipment correlation coefficient is 10.

[0058] It should be noted that, assuming the two sets of correlated images from different categories contain the same camera, the difference lies in the order in which the cameras are positioned. The corresponding device correlation coefficients may be the same or different. For example, when the same person approaches from different directions, some cameras may not cover the person's path, resulting in one camera capturing the person while another fails to do so. For instance, in one set of correlated images, the first camera is c. u The device in the latter is c i The parameter r in the device association matrix is ui Another type of associated image group is c in the front device. i The device in the latter is c u The parameter r in the device association matrix is iu Then parameter r ui and parameter r iu They may be the same, or they may be different.

[0059] In this embodiment, the device correlation coefficients between shooting devices within a preset area are calculated using pre-collected comparison images of personnel and sample images, forming a device correlation matrix that characterizes the degree of correlation between shooting devices. This provides effective data for subsequent correction of the image similarity of the images to be corrected based on the device correlation matrix, thereby improving the accuracy of image similarity correction and ultimately increasing the accuracy of image recall.

[0060] In one embodiment, generating a device association matrix based on all associated device groups includes:

[0061] (1) Obtain the initial correlation matrix; the initial correlation matrix includes multiple matrix coefficients; the matrix coefficients represent the number of times the same sample person is photographed by a subsequent device within a preset device correlation time period when the previous device photographed the person.

[0062] Understandably, the above description assumes the existence of a parameter r in the device association matrix. ui (i.e., matrix coefficients), this parameter represents the position of the person in the shooting device c. u The captured device c is associated with the device within the time t after the snapshot. iThe captured statistics. Therefore, within the preset area, there are two matrix coefficients between every two capturing devices, the only difference between these two matrix coefficients being the order in which the two devices captured the images. For example, r ui and r iu Furthermore, the "forward device" refers to the device that captured images of the same sample of people earlier in the photographic process. The "backward device" refers to the device that captured images of the same sample of people later in the photographic process. Furthermore, all matrix coefficients in the initial correlation matrix currently correspond to a value of zero.

[0063] (2) In the same associated device group, the shooting device corresponding to the first matched image is determined as the first device, and the shooting device corresponding to the second matched image is determined as the second device.

[0064] Understandably, the above description indicates that there are two images in the associated device group: the preceding matched image and the following matched image. The preceding matched image and the following matched image originate from different shooting devices. Therefore, the shooting device corresponding to the preceding matched image can be identified as the preceding device, and the shooting device corresponding to the following matched image can be identified as the following device.

[0065] (3) The associated device group with the same preceding and following devices as the matrix coefficient is determined as the matching device group corresponding to the matrix coefficient.

[0066] Specifically, after determining the capturing device corresponding to the preceding matched image in the same associated device group as the preceding device and the capturing device corresponding to the following matched image as the following device, the preceding and following devices corresponding to each matrix coefficient can be matched with the preceding and following devices in the associated device group. Thus, the associated device group that matches the preceding and following devices corresponding to the matrix coefficient is determined as the matching device group corresponding to that matrix coefficient.

[0067] (4) Adjust the matrix coefficients based on the total number of matching device groups corresponding to the matrix coefficients, and determine the adjusted initial association matrix as the device association matrix.

[0068] Specifically, after identifying the associated device groups with the same preceding and following devices as the matrix coefficients as the matching device groups corresponding to the matrix coefficients, the total number of matching device groups corresponding to each matrix coefficient can be counted, and the total number corresponding to each matrix coefficient can be determined as the value of the matrix coefficient. After all the values ​​corresponding to the matrix coefficients have been filled and adjusted, the adjusted initial association matrix can be determined as the device association matrix.

[0069] For example, assuming that the shooting device corresponding to the first matched image in the associated device group is c2 and the shooting device corresponding to the second matched image is c4, then the matrix coefficient in the initial association matrix is ​​r24. If there are 5 associated device groups corresponding to r24, then this parameter is incremented by 5.

[0070] In one embodiment, before obtaining the preset device association time, the method further includes:

[0071] (1) Obtain the total shooting time of each shooting device within the preset area when the number of images taken by the same shooting person reaches the preset number.

[0072] (2) Determine the device association time based on the total shooting time spent by each shooting device in the preset area for the same shooting person when the number of images reaches the preset number.

[0073] Understandably, the preset number of times can be set according to needs. For example, the preset number of times can be set to 5, 6, or 7 times, etc. Specifically, an experiment can be conducted within a preset area. For example, suppose there are five shooting devices in the preset area, and then the target object (such as a person or robot, or other movable object) starts walking from the first shooting device and stops after reaching the last shooting device. Then, the total shooting time of each shooting device taking five consecutive shots of the target object is collected. Finally, the ratio between the sum of the total shooting time of all shooting devices and the total number of shooting devices is determined as the device association time. Generally, the device association time is about 30 minutes on urban roads and about 20 minutes in parks.

[0074] In one embodiment, step S30, namely, correcting the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity corresponding to the image to be corrected, includes:

[0075] (1) Select one image from all the images to be corrected as the third target image.

[0076] Understandably, the third target image is one of the images to be corrected selected from all the images to be corrected. The selection method can be random selection or sequential selection, etc.

[0077] (2) Obtain the device association time, and select the first target image with a time interval less than or equal to the preset device association time from all the first target images as the fourth target image; the time interval refers to the interval between the shooting time of the first target image and the shooting time of the third target image.

[0078] Understandably, the device association time mentioned above can be determined based on the density and capture rate of the shooting devices distributed within the preset area. Therefore, after selecting the third target image, images to be corrected captured within the device association time prior to the third target image can be selected as preceding images, and images to be corrected captured within the device association time after the third target image can be selected as following images. All preceding images and all following images constitute all the selected fourth target images.

[0079] (3) Obtain the preset correction coefficients and determine the image correction coefficients corresponding to each fourth target image based on the preset correction coefficients.

[0080] Understandably, the preset correction factor refers to the adjusted appearance of the target person on the shooting device c. u The video is captured by device c during the subsequent device association time t. i The probability of capturing the image. Specifically, after selecting the preceding and following images corresponding to the third target image from the preset image selection sequence, a preset correction coefficient is obtained. Then, based on the preset correction coefficient, the image correction coefficient corresponding to each fourth target image is determined. As mentioned above, the fourth target image can be divided into preceding and following images. Furthermore, the preceding coefficient can be determined based on the device correlation degree between the shooting devices corresponding to different preceding images and the shooting devices corresponding to the third target image in the device correlation matrix, and the preset correction coefficient; and the following coefficient can be determined based on the device correlation degree between the shooting devices corresponding to different following images and the shooting devices corresponding to the third target image in the device correlation matrix.

[0081] The coefficient in front is:

[0082]

[0083] Wherein, rtu refers to the imaging device c corresponding to the third target image. u The statistics of images captured by the camera device ct within the device association time t before the capture. This refers to the imaging device c corresponding to the third target image. u The total number of images captured by the device association matrix within the device association time t prior to the capture, from the capturing device corresponding to the first preceding image to the capturing device corresponding to the p2th preceding image. α is the Laplace smoothing coefficient; k refers to the number of columns in the device association matrix.

[0084] The coefficient is then:

[0085]

[0086] Where, r ui This refers to the imaging device c corresponding to the third target image.u The captured device c is associated with the device within the time t after the snapshot. i Statistics captured. This refers to the imaging device c corresponding to the third target image. u The total number of images captured within the device association time t after capture, starting from the capturing device corresponding to the first subsequent image to the capturing device corresponding to the p1th subsequent image. α is the Laplace smoothing coefficient; k refers to the number of columns in the device association matrix.

[0087] (4) Based on all image correction coefficients, preset similarity thresholds and the similarity of the third target image, determine the corrected similarity of the third target image.

[0088] Specifically, after determining the preceding and following coefficients, the corrected similarity of the third target image can be determined based on all image correction coefficients, a preset similarity threshold, and the similarity to the third target image. In other words, the corrected similarity of the third target image is determined based on all preceding coefficients, all following coefficients, a preset similarity threshold, and the similarity to the third target image.

[0089] (5) Select a new image from all the images to be corrected as the new third target image. The new third target image is the image that has not been selected from all the images to be corrected.

[0090] (6) Continue to determine the corrected similarity corresponding to the new third target image until all images to be corrected have been selected, so as to obtain the corrected similarity corresponding to all images to be corrected.

[0091] Specifically, after determining the corrected similarity of the third target image, an unselected image from all images to be corrected is chosen as the new third target image. The corrected similarity of the new third target image is then calculated using the steps described above. The method for determining the corrected similarity is the same and will not be repeated here. Once all images to be corrected have been selected, it signifies that the image similarity of all images to be corrected has been corrected.

[0092] In this embodiment, by introducing a device association matrix, the image similarity of the image to be corrected is corrected by leveraging the device association degree between the shooting devices corresponding to the image to be corrected in the device association matrix. This allows the image similarity of the image to be corrected that is closer to the target person to approach a preset similarity threshold, thereby improving the recall rate of the captured image.

[0093] In one embodiment, the preceding and following images corresponding to the first target image can be determined in the following manner:

[0094] (1) Obtain the first shooting time corresponding to the third target image, and the second shooting time corresponding to the other images to be corrected besides the third target image.

[0095] Understandably, each image captured by the imaging device has a corresponding capture time, which can be directly obtained from the database associated with the imaging device. The first capture time is the time when the imaging device captures the third target image. The second capture time is the second capture time corresponding to the capture of other images to be corrected besides the third target image. Each other image to be corrected besides the third target image corresponds to one second capture time.

[0096] (2) The time point corresponding to the device association time before the first shooting time is determined as the previous shooting time, and the time point corresponding to the device association time after the first shooting time is determined as the later shooting time.

[0097] Understandably, the "pre-capture time" refers to the time point corresponding to the device association time before the first capture time. The "sub-capture time" refers to the time point corresponding to the device association time after the first capture time. For example, assuming the first capture time is 3:30 PM on September 3, 2022, and the device association time is 3 minutes, then the corresponding "pre-capture time" is 3:27 PM on September 3, 2022, and the corresponding "sub-capture time" is 3:33 PM on September 3, 2022.

[0098] (3) The image to be corrected corresponding to the second shooting time in the preceding time range is determined as the preceding image; the preceding time range is between the preceding shooting time and the first shooting time.

[0099] Specifically, after determining the previous shooting time, the time range between the previous shooting time and the first shooting time is defined as the previous time range. Then, the second shooting time is compared with the previous time range, and the image to be corrected corresponding to the second shooting time within the previous time range is determined as the previous image. That is, the second shooting time corresponding to the previous image is later than the previous shooting time but earlier than the first shooting time.

[0100] (4) The image to be corrected corresponding to the second shooting time in the later time range is determined as the later image; the earlier time range is between the first shooting time and the later shooting time.

[0101] Specifically, after determining the subsequent shooting time, the time range between the first shooting time and the subsequent shooting time is defined as the subsequent time range. Then, the second shooting time is compared with the subsequent time range, and the image to be corrected corresponding to the second shooting time within the subsequent time range is determined as the subsequent image. That is, the second shooting time corresponding to the subsequent image is later than the first shooting time but earlier than the subsequent shooting time.

[0102] In one embodiment, determining the corrected similarity of the third target image based on all image correction coefficients, a preset similarity threshold, and the similarity to the third target image includes:

[0103] Understandably, the above description indicates that the fourth target image may include the preceding image, and thus the image correction coefficient corresponding to the preceding image is the preceding coefficient. Furthermore, the preceding similarity is determined based on a preset similarity threshold, the image similarity corresponding to the third target image, and all preceding coefficients.

[0104] For expressions related to prior similarity:

[0105]

[0106] Where ft refers to the preceding similarity of the t-th preceding image. rtu refers to the similarity of the image captured by the third target image. u The statistics of images captured by the camera device ct within the device association time t before the capture. This refers to the imaging device c corresponding to the third target image. u The total number of images captured by the device association matrix within the time t before the capture, from the first image corresponding to the capturing device to the second image corresponding to the capturing device, where k is the number of columns in the device association matrix, s is the image similarity of the image to be corrected, and s1 is the preset similarity threshold.

[0107] Understandably, the above description indicates that the fourth target image also includes subsequent images, and therefore the image correction coefficients corresponding to the subsequent images are the front-to-back coefficients. Furthermore, based on a preset similarity threshold, the image similarity corresponding to the third target image, and all subsequent coefficients, the subsequent similarity is determined.

[0108] For expressions related to post-similarity:

[0109]

[0110] Where fi refers to the subsequent similarity of the i-th subsequent image. s1 refers to the preset similarity threshold; s refers to the image similarity of the third target image; r ui This refers to the imaging device c corresponding to the third target image. uThe captured device c is associated with the device within the time t after the snapshot. i Statistics captured. This refers to the imaging device c corresponding to the third target image. u The total number of images captured within the device association time t after capture, from the capturing device corresponding to the first subsequent image to the capturing device corresponding to the p1th subsequent image. α is the Laplacian smoothing coefficient; k is the number of columns in the device association matrix; s is the image similarity of the image to be corrected; and s1 is the preset similarity threshold.

[0111] The corrected similarity of the third target image is determined based on the preceding and following similarities.

[0112] Specifically, after determining the preceding similarity of the preceding image and the following similarity of the following image, all preceding and following similarities can be accumulated. A correction value is determined by multiplying the difference between the preset similarity threshold and the image similarity corresponding to the third target image, the sum of all preceding similarities, and the sum of all following similarities. The corrected similarity of the third target image is then determined by summing the image similarity corresponding to the third target image and the correction value. This is illustrated in the following expression:

[0113]

[0114] Where F is the corrected similarity corresponding to the third target image, s is the image similarity of the image to be corrected, and s1 is the preset similarity threshold.

[0115] In one embodiment, obtaining a preset correction coefficient includes:

[0116] (1) Obtain the shooting reproduction probability coefficient and smooth the shooting reproduction probability coefficient to obtain the smoothed reproduction probability coefficient; the smoothed reproduction probability coefficient includes the smoothing sub-parameter.

[0117] Understandably, the probability coefficient of reproducibility refers to the probability of the target person appearing on the shooting device c. u The video is captured by device c during the subsequent device association time t. i The probability of capturing an image. Since the sample images used in the initial construction of the device association matrix are relatively scarce, this embodiment introduces the Laplacian smoothing method to smooth the image recurrence probability coefficients, resulting in smoothed recurrence probability coefficients. Specifically:

[0118]

[0119]

[0120] Wherein, equation (1) is the expression for the probability coefficient of image reproduction; equation (2) is the expression for the probability coefficient of smooth reproduction; r ui The device c refers to the person being filmed. u The video is captured by device c during the subsequent device association time t. i Statistical data captured; r un The device c refers to the person being filmed. u The video was captured and then captured by other shooting devices (including shooting device c) within the subsequent device association time t. i The captured presidential measurement (n refers to the nth capturing device, and k refers to the number of columns in the device association matrix). α is the smoothing sub-parameter.

[0121] (2) Adjust the smoothing sub-parameters according to the correlation coefficients of all equipment in the equipment correlation matrix to obtain the correction sub-parameters.

[0122] Understandably, as stated above, the device correlation coefficient is determined by the number of associated image groups corresponding to the preceding and following devices; that is, all device correlation coefficients reflect the total number of associated image groups contained in the device correlation matrix. The statement above also indicates that when associated image groups exist, the device correlation coefficient corresponding to the associated image group is incremented by 1, i.e., the sample count is incremented by 1. Therefore, the number of samples contained in the device correlation matrix can be directly queried from each column or row of the device correlation matrix. Based on the sample count, the smoothing sub-parameter in the smoothed recurrence probability coefficient is adjusted to obtain the corrected sub-parameter. The adjusted Laplace coefficient is as follows:

[0123]

[0124] Where α′ is the correction sub-parameter; α is the smoothing sub-parameter; rmn is the device correlation coefficient in the m-th row and n-th column; and b is the adjustment coefficient.

[0125] (3) Replace the smoothing sub-parameter in the smooth recurrence probability coefficient with the correction sub-parameter, and determine the replaced smooth recurrence probability coefficient as the preset correction coefficient.

[0126] Specifically, after adjusting the smoothing sub-parameter based on the correlation coefficients of all devices in the device correlation matrix to obtain the correction sub-parameter, the correction sub-parameter replaces the smoothing sub-parameter in the smoothing recurrence probability coefficient, and the replaced smoothing recurrence probability coefficient is determined as the preset correction coefficient.

[0127] In this embodiment, to avoid potential bias or inaccuracies in the initial device association matrix due to the scarcity of sample images, a Laplacian smoothing method is introduced to smooth the image reproduction probability coefficients, thereby improving the accuracy of similarity correction. However, considering that setting the Laplacian coefficient too high could lead to excessively large corrections for image similarity between densely packed shooting devices, the Laplacian coefficient is further adjusted to further improve the accuracy of similarity correction.

[0128] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0129] In one embodiment, an image retrieval device is provided, which corresponds one-to-one with the image retrieval methods described in the above embodiments. For example... Figure 3 As shown, the image retrieval device includes an image acquisition module 10, an image segmentation module 20, a similarity correction module 30, and a retrieved image determination module 40. Detailed descriptions of each functional module are as follows:

[0130] Image acquisition module 10 is used to acquire the target person image corresponding to the target person and the captured image obtained by the shooting device within the preset area; one shooting device corresponds to at least one captured image;

[0131] The image segmentation module 20 is used to determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, to segment the captured image into a first target image and an image to be corrected;

[0132] The similarity correction module 30 is used to obtain the device association matrix corresponding to the shooting devices in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity.

[0133] The recall image determination module 40 is used to filter out a second target image from the images to be corrected based on the corrected similarity, and use the first target image and the second target image as the recall images corresponding to the target person.

[0134] Preferably, the image segmentation module 20 includes:

[0135] A similarity comparison unit is used to compare the image similarity with the preset similarity threshold;

[0136] The first image segmentation unit is used to identify captured images with an image similarity less than the preset similarity threshold as the images to be corrected;

[0137] The second image segmentation unit is used to identify captured images with an image similarity greater than or equal to the similarity threshold as the first target image.

[0138] Preferably, the similarity correction module 30 includes:

[0139] The image acquisition unit is used to acquire the comparison image of the sample personnel and the sample image captured by the shooting device in the preset area; one shooting device corresponds to at least one sample image; one sample image corresponds to one sample shooting time;

[0140] A similarity determination unit is used to determine the sample similarity between the comparison personnel image and the sample image, and to determine the sample image corresponding to the sample similarity greater than the preset similarity threshold as the sample matching image, thereby obtaining multiple sample matching images;

[0141] The correlation coefficient determination unit is used to determine the device correlation coefficient between any two shooting devices within the preset area based on the shooting time of each of the sample matching images and the shooting device information to which each of the sample matching images belongs.

[0142] The matrix generation unit is used to determine the device association matrix corresponding to the shooting devices in the preset area based on the device association coefficient between any two shooting devices in the preset area.

[0143] Preferably, the correlation coefficient determination unit includes:

[0144] The time comparison subunit is used to compare the time difference between the shooting times of any two sample matching images, and to determine two sample matching images whose time difference is less than a preset device association time as an associated image group, thereby obtaining multiple associated image groups.

[0145] The device area molecular unit is used to determine, for each of the associated image groups, the capturing device to which the sample matching image captured earlier in the associated image group belongs as the preceding device and the capturing device to which the sample matching image captured later in the associated image group belongs as the following device.

[0146] The device classification subunit is used to classify multiple associated image groups according to the shooting device information of the two sample matching images in the associated image group, and obtain the associated image groups corresponding to the same pair of the preceding device and the following device respectively.

[0147] The coefficient determination subunit is used to determine the device association coefficient between any two shooting devices within the preset area based on the number of associated image groups corresponding to each pair of the front device and the rear device.

[0148] Preferably, the image retrieval device further includes

[0149] The shooting time determination module is used to obtain the total shooting time spent by each shooting device in the preset area for the same shooting person when the number of images captured reaches a preset number.

[0150] The association time determination module is used to determine the device association time based on the total shooting time spent by each shooting device in the preset area for the same shooting person when the number of images taken reaches a preset number.

[0151] Preferably, the similarity correction module 30 further includes:

[0152] The third target image selection unit is used to select one image to be corrected from all the images to be corrected as the third target image;

[0153] The first image selection unit is used to obtain a preset device association time and select a first target image from all the first target images whose time interval is less than or equal to the preset device association time as the fourth target image; the time interval refers to the interval between the shooting time of the first target image and the shooting time of the third target image;

[0154] A coefficient determination unit is used to obtain a preset correction coefficient and determine the image correction coefficient corresponding to each of the fourth target images based on the preset correction coefficient.

[0155] The first similarity correction unit is used to determine the corrected similarity of the third target image based on all the image correction coefficients, the preset similarity threshold and the similarity of the third target image;

[0156] The second image selection unit is used to reselect one image to be corrected from all images to be corrected as a new third target image, wherein the new third target image is an image to be corrected that has not been selected from all images to be corrected.

[0157] The second similarity correction unit is used to continue to determine the corrected similarity corresponding to the new third target image until all the images to be corrected have been selected, so as to obtain the corrected similarity corresponding to all the images to be corrected.

[0158] Preferably, the coefficient determination unit includes:

[0159] The coefficient acquisition subunit is used to acquire the shooting reproduction probability coefficient and smooth the shooting reproduction probability coefficient to obtain the smoothed reproduction probability coefficient; the smoothed reproduction probability coefficient includes a smoothing sub-parameter.

[0160] The coefficient adjustment subunit is used to adjust the smoothing sub-parameter according to the correlation coefficients of all devices in the device correlation matrix to obtain the correction sub-parameter;

[0161] The coefficient replacement subunit is used to replace the smoothing sub-parameter in the smooth recurrence probability coefficient with the correction sub-parameter, and to determine the replaced smooth recurrence probability coefficient as the preset correction coefficient.

[0162] For specific limitations regarding the image retrieval device, please refer to the limitations of the image retrieval method above, which will not be repeated here. Each module in the aforementioned image retrieval device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0163] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data used in the image retrieval method described in the above embodiments. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements an image retrieval method.

[0164] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image retrieval method described in the above embodiment.

[0165] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image retrieval method described above.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0167] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0168] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An image recall method, characterized in that, include: Acquire images of the target personnel and images captured by cameras within a preset area; One of the shooting devices corresponds to at least one of the captured images; Determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, divide the captured image into a first target image and an image to be corrected; Obtain the device association matrix corresponding to the shooting devices in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity corresponding to the image to be corrected; Based on the corrected similarity, a second target image is selected from the images to be corrected, and the first target image and the second target image are used as the recall images corresponding to the target person; The step of obtaining the device association matrix corresponding to the shooting devices within the preset area includes: The system acquires images of the comparison personnel corresponding to the sample personnel and sample images captured by the imaging device within the preset area; each imaging device corresponds to at least one sample image; each sample image corresponds to one sample capture time. Determine the sample similarity between the comparison personnel image and the sample image, and identify all sample images with a sample similarity greater than the preset similarity threshold as sample matching images to obtain multiple sample matching images; Based on the shooting time of each sample matching image and the shooting device information to which each sample matching image belongs, the device correlation coefficient between any two shooting devices within the preset area is determined; Based on the device association coefficient between any two shooting devices within the preset area, determine the device association matrix corresponding to the shooting devices within the preset area; The step of correcting the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity corresponding to the image to be corrected includes: Select one image from all the images to be corrected as the third target image; A preset device association time is obtained, and a first target image with a time interval less than or equal to the preset device association time is selected from all the first target images as the fourth target image; the time interval refers to the interval between the shooting time of the first target image and the shooting time of the third target image; Obtain a preset correction coefficient, and determine the image correction coefficient corresponding to each of the fourth target images based on the preset correction coefficient; Based on all the image correction coefficients, the preset similarity threshold, and the similarity corresponding to the third target image, the corrected similarity corresponding to the third target image is determined; A new third target image is selected from all the images to be corrected, which is an image that has not been selected from all the images to be corrected. Continue to determine the corrected similarity corresponding to the new third target image until all the images to be corrected have been selected, so as to obtain the corrected similarity corresponding to all the images to be corrected.

2. The image recall method as described in claim 1, characterized in that, The step of dividing the captured image into a first target image and an image to be corrected based on the image similarity and a preset similarity threshold includes: The image similarity is compared with the preset similarity threshold; Images with an image similarity less than the preset similarity threshold are identified as the images to be corrected; Images with an image similarity greater than or equal to the preset similarity threshold are identified as the first target image.

3. The image retrieval method as described in claim 1, characterized in that, The step of determining the device correlation coefficient between any two shooting devices within the preset area based on the shooting time of each of the sample matching images and the shooting device information to which each sample matching image belongs includes: Compare the time difference between the shooting times of any two sample matching images, and determine the two sample matching images whose time difference is less than the preset device association time as an associated image group to obtain multiple associated image groups; For each of the associated image groups, the capturing device to which the sample matching image with the earlier capture time belongs in the associated image group is determined as the earlier capturing device, and the capturing device to which the sample matching image with the later capture time belongs in the associated image group is determined as the later capturing device. Based on the shooting device information of the two sample matching images in the associated image group, multiple associated image groups are classified to obtain the associated image groups corresponding to the same pair of front and back devices. Based on the number of associated image groups corresponding to each pair of the front device and the rear device, the device association coefficient between any two shooting devices within the preset area is determined.

4. The image recall method as described in claim 1, characterized in that, Before obtaining the preset device association time, the process also includes: Obtain the total shooting time of each shooting device within the preset area when the number of images taken by the same photographer reaches a preset number; The device association time is determined based on the total shooting time spent by each shooting device within the preset area for the same photographer when the number of images captured reaches a preset number.

5. The image retrieval method as described in claim 1, characterized in that, The process of obtaining the preset correction coefficient includes: The shooting reproduction probability coefficient is obtained, and the shooting reproduction probability coefficient is smoothed to obtain the smoothed reproduction probability coefficient; the smoothed reproduction probability coefficient includes a smoothing sub-parameter. Based on the correlation coefficients of all devices in the device correlation matrix, the smoothing sub-parameter is adjusted to obtain the correction sub-parameter; Replace the smoothing sub-parameter in the smooth recurrence probability coefficient with the correction sub-parameter, and determine the replaced smooth recurrence probability coefficient as the preset correction coefficient.

6. An image recall device, characterized in that, include: The image acquisition module is used to acquire the target person's image and the captured image obtained by the shooting device within the preset area; One of the shooting devices corresponds to at least one of the captured images; An image segmentation module is used to determine the image similarity between the target person image and the captured image, and based on the image similarity and a preset similarity threshold, to segment the captured image into a first target image and an image to be corrected; The similarity correction module is used to obtain the device association matrix corresponding to the shooting devices in the preset area, and correct the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity. The recall image determination module is used to filter out a second target image from the images to be corrected based on the corrected similarity, and use the first target image and the second target image as the recall images corresponding to the target person; The step of obtaining the device association matrix corresponding to the shooting devices within the preset area includes: The system acquires images of the comparison personnel corresponding to the sample personnel and sample images captured by the imaging device within the preset area; each imaging device corresponds to at least one sample image; each sample image corresponds to one sample capture time. Determine the sample similarity between the comparison personnel image and the sample image, and identify all sample images with a sample similarity greater than the preset similarity threshold as sample matching images to obtain multiple sample matching images; Based on the shooting time of each sample matching image and the shooting device information to which each sample matching image belongs, the device correlation coefficient between any two shooting devices within the preset area is determined; Based on the device association coefficient between any two shooting devices within the preset area, determine the device association matrix corresponding to the shooting devices within the preset area; The step of correcting the image similarity of the image to be corrected based on the device association matrix to obtain the corrected similarity corresponding to the image to be corrected includes: Select one image from all the images to be corrected as the third target image; A preset device association time is obtained, and a first target image with a time interval less than or equal to the preset device association time is selected from all the first target images as the fourth target image; the time interval refers to the interval between the shooting time of the first target image and the shooting time of the third target image; Obtain a preset correction coefficient, and determine the image correction coefficient corresponding to each of the fourth target images based on the preset correction coefficient; Based on all the image correction coefficients, the preset similarity threshold, and the similarity corresponding to the third target image, the corrected similarity corresponding to the third target image is determined; A new third target image is selected from all the images to be corrected, which is an image that has not been selected from all the images to be corrected. Continue to determine the corrected similarity corresponding to the new third target image until all the images to be corrected have been selected, so as to obtain the corrected similarity corresponding to all the images to be corrected.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image retrieval method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image retrieval method as described in any one of claims 1 to 5.

Images