Image archiving method, device, computer equipment and computer readable storage medium

Abstract

The application relates to the technical field of image processing, in particular to an image archiving method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: acquiring an image connection network constructed based on N images to be archived; determining to-be-occupied memory based on the number of nodes in the image connection network and the number of edges in the image connection network; dividing the N images to be archived into M piece sets according to the current remaining memory and the to-be-occupied memory; clustering the M piece sets respectively to obtain M clustering results, each piece set corresponding to a clustering result; and obtaining an archiving result of the N images to be archived based on the M clustering results. The batch number is related to the complexity of the data set, can be applied to the data set in any case, has good applicability, and can effectively improve the rational utilization rate of memory compared with the fixed batch number.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image archiving method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] Currently, target tracking technologies based on image or video streams can automatically complete target tracking, reducing manual labor and offering high accuracy and efficiency. In tracking scenarios, they can achieve good tracking results to a certain extent. However, with the widespread use of high-definition cameras and the need for centralized processing of large amounts of historical data, significant processor memory is required during processing. For example, in face or body tracking scenarios, the InfoMap algorithm is used for community detection to obtain clustering results. Associating the face and body clustering results with each cluster yields the final archive set. Since the InfoMap algorithm requires graph construction for clustering and archiving, memory usage increases with the data volume. Therefore, when memory is insufficient for large data volumes, data sharding is necessary. Currently, sharding at a fixed scale and then performing batch clustering and archiving can solve this problem to some extent. However, current sharding algorithms use fixed-scale sharding. Using the same scale of sharding for different types of datasets results in varying memory usage during actual processing, potentially leading to low memory utilization in some cases. Therefore, how to reasonably configure the number of sharding operations and avoid using sharding with a fixed amount of data in order to improve the reasonable utilization of memory has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of this application provide an image archiving method, apparatus, computer device, and computer-readable storage medium to solve the problem of reasonably configuring the number of fragments and avoiding the use of fixed data volume fragments, so as to improve the reasonable utilization of memory.

[0004] In a first aspect, embodiments of this application provide an image archiving method, the image archiving method comprising:

[0005] An image connection network is constructed based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to characterize the correlation between two nodes of the edge, which satisfies a preset condition. Here, N is an integer greater than one.

[0006] The memory to be occupied is determined based on the number of nodes in the image connection network and the number of edges in the image connection network.

[0007] Based on the current remaining memory and the memory to be occupied, the N images to be archived are divided into M fragment sets, each fragment set including at least one image to be archived, where M is a positive integer;

[0008] Based on the M fragment sets, the archiving results of the N images to be archived are obtained.

[0009] Secondly, embodiments of this application provide an image archiving device, the image archiving device comprising:

[0010] A network construction module is used to construct an image connection network based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to characterize the correlation between two nodes of the edge, which satisfies a preset condition. Here, N is an integer greater than one.

[0011] The memory determination module is used to determine the memory to be occupied based on the number of nodes in the image connection network and the number of edges in the image connection network.

[0012] The sharding processing module is used to divide the N images to be archived into M shard sets according to the current remaining memory and the memory to be occupied, where each shard set includes at least one image to be archived, and M is a positive integer;

[0013] The fragment archiving module is used to obtain the archiving results of the N images to be archived based on the M fragment sets.

[0014] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image archiving method as described in the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image archiving method as described in the first aspect.

[0016] The beneficial effects of the embodiments in this application compared with the prior art are:

[0017] An image connection network is constructed based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network represent the correlation between two nodes of the edge, satisfying a preset condition. Based on the number of nodes and edges in the image connection network, the memory to be occupied is determined. According to the current remaining memory and the memory to be occupied, the N images to be archived are divided into M fragment sets, each fragment set including at least one image to be archived. Based on the M fragment sets, the archiving result of the N images to be archived is obtained. Specifically, using the images to be archived as nodes, the complexity of the dataset of N images to be archived is described through the correlation of nodes and edges, thereby determining the required memory size. Simultaneously, the remaining memory guides the fragmentation of the N images to be archived, enabling clustering, fusion archiving, and other operations in different batches, thus obtaining the archiving result. The number of batches is related to the complexity of the dataset and can be applied to datasets in any situation, exhibiting good applicability. Compared to a fixed number of batches, it can effectively improve the rational utilization of memory. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an application environment for an image archiving method provided in Embodiment 1 of this application;

[0020] Figure 2 This is a flowchart illustrating an image archiving method provided in Embodiment 1 of this application;

[0021] Figure 3 This is a flowchart illustrating an image archiving method provided in Embodiment 2 of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an image archiving device provided in Embodiment 3 of this application;

[0023] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0028] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0031] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0032] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0033] To illustrate the technical solution of this application, specific embodiments are described below.

[0034] The image archiving method provided in Embodiment 1 of this application can be applied to, for example, Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0035] See Figure 2 This is a flowchart illustrating an image archiving method provided in Embodiment 1 of this application. The image archiving method described above can be applied to... Figure 1The server-side component connects to the client, database, and other devices to acquire relevant data. For example, the client might be an image acquisition device that sends real-time images to the server for archiving. The image archiving method described above, implemented on the server, can be applied to scenarios such as target tracking and localization in images. This process requires image processing methods such as image recognition and classification. AI can be used to perform specific image processing, but this paper focuses on utilizing the results of image processing methods for segmentation, clustering, and archiving, which will be detailed below.

[0036] like Figure 2 As shown, the image archiving method may include the following steps:

[0037] Step S201: Construct an image connectivity network based on N images to be archived.

[0038] In this application, the image to be archived is acquired using a corresponding image acquisition device. For example, the image acquisition device is installed and aligned with the area to be detected, thereby acquiring information within that area to form an image. The client connected to the server in this application can be a user's terminal device. The user uses this terminal device to upload images acquired by an image acquisition device connected to it. Alternatively, the client can be compatible with the image acquisition function of the image acquisition device, and the user uses the image acquisition function of the terminal device to acquire images and then sends them to the server. Of course, the client can also be the image acquisition device directly, that is, the server directly connects to the image acquisition device to directly obtain the corresponding image. In addition, the server can also connect to a database to obtain images stored in the database as images to be archived. All of the above are methods of obtaining images to be archived, and this application does not limit them. Wherein, N is an integer greater than one, specifically a large value, which can reflect the optimization effect of this application for processing large amounts of data.

[0039] The specific targets defined in the images to be archived vary depending on the application scenario. For example, in a face tracking scenario, the information in the images to be archived should be facial information. In a vehicle tracking scenario, the information in the images to be archived should be vehicle appearance, license plate, etc. The person corresponding to the face, the vehicle corresponding to the appearance, etc., are the targets in the images to be archived. Multiple people's faces and multiple vehicle appearances may exist in N images to be archived. Taking the target as a person as an example, the purpose of archiving is to group images corresponding to the same person's face into the same archive.

[0040] In archiving, clustering is generally used, for example, the InfoMap algorithm is used for community detection to cluster images. In this application, the nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to represent the correlation between two nodes of the edge, which meets the preset conditions. The correlation between two nodes can be obtained by calculating the similarity between the images to be archived corresponding to the two nodes. For example, the image information of the images to be archived is used as the node information, and the correlation between the node information is calculated to construct the edges. Here, the image information can refer to the original pixel values, grayscale pixel values, and other information represented by the image to be archived, or it can be the feature values ​​extracted by the encoder. If feature values ​​are used, the encoder in the neural network model needs to be used for processing, and it can achieve better clustering results in the subsequent clustering, including improving the efficiency and accuracy of similarity calculation.

[0041] By using nodes and node information, N images to be archived can be converted into a graph-structured data representation, i.e., an image connection network, reducing redundant information and facilitating clustering.

[0042] In this application, within the image connection network corresponding to N images to be archived, a connection table is needed to represent the connection relationships between nodes, that is, to represent whether two images to be archived are related. This relatedness can be based on similarity in information content, space, time, etc., and determining the relatedness requires relying on the node information. For any two nodes, such as the first node and the second node, the similarity between the node information of the first node and the node information of the second node needs to be calculated to determine whether the two node information is similar in information content, time, and space.

[0043] In this embodiment, the similarity of information content is taken as an example. Specifically, it determines whether the images to be archived corresponding to two nodes are similar. The similarity between images can be obtained through similarity calculation methods such as Euclidean distance and cosine similarity. Combined with a preset threshold, images are judged to be similar if the similarity is greater than the threshold, and dissimilar if the similarity is less than the threshold. Specifically, in the case of similarity, the association between the two nodes meets a preset condition. In this case, a connecting edge is constructed between the two nodes to represent the connection between them, indicating that they are similar and can be grouped into the same cluster. In the case of dissimilarity, the association between the two nodes does not meet the preset condition, meaning they cannot be grouped into the same cluster.

[0044] In addition to considering the similarity between images in practical applications, similarity in time and space is also calculated to enhance the judgment of correlation. For example, if two images are acquired at similar times, it can help determine that the information content represented by the two images should be similar. If two images are acquired at different spaces but are relatively close in time, it can help determine that the information content represented by the two images should be dissimilar.

[0045] For example, in a face tracking scenario, images representing the same person are related. Therefore, during clustering, it is necessary to group the two into one class. This requires building connecting edges between corresponding nodes. Therefore, using the similarity of node information, i.e. image information, to determine whether to build connecting edges can effectively meet the above requirements, thereby achieving the tracking of the person.

[0046] Step S202: Determine the memory to be occupied based on the number of nodes in the image connection network and the number of edges in the image connection network.

[0047] In this application, the number of nodes can be used to express the number of images to be archived, and the number of edges of a node can express the number of other images associated with an image to be archived. For a node, the number of edges represents the complexity of the node during processing, and thus the memory required for processing (i.e., the memory to be occupied) can be determined. It can be seen that the size of the memory to be occupied is directly proportional to the complexity, while the number of nodes and the number of edges are directly proportional to the complexity. Therefore, the memory to be occupied is positively correlated with the number of nodes and the number of edges.

[0048] The total number of nodes in the above statistics should be the same as the number of images to be archived. An edge between two nodes can be recorded as one edge during the statistics process. For ease of counting, each node's edge can be counted separately, meaning the same edge will be recorded twice. Therefore, the recorded number of edges is twice the actual number of edges.

[0049] Optionally, the memory to be occupied is determined based on the number of nodes and the number of edges in the image connectivity network, including:

[0050] Obtain the first and second positive parameters that are on the same order of magnitude as the number of adjustment nodes and the number of edges, and obtain the third positive parameter that is on the same order of magnitude as the number of memory.

[0051] Based on the first positive parameter, the second positive parameter, the number of nodes, and the number of edges, the quantity to be calculated is obtained. Based on the third positive parameter and the quantity to be calculated, the memory to be occupied is obtained.

[0052] The number of nodes is roughly equivalent to the number of images to be archived, and the number of edges is generally no greater than the number of images to be archived. If the number of images to be archived is large, the number of nodes and the number of edges may differ significantly, for example, by a factor of ten.

[0053] To facilitate the explanation of the relationship between memory, the number of nodes, and the number of edges, the first positive parameter adjusts the order of magnitude of the number of nodes to be the same as the order of magnitude of the number of edges adjusted by the second positive parameter. For example, if the number of nodes is 10... 3 The number of edges is correspondingly 10 2 and 10 4 Between. For the same purpose, a third positive parameter is used to adjust the number of computations obtained based on the first positive parameter, the second positive parameter, the number of nodes, and the number of edges to be on the same order of magnitude as the memory. For example, if the total memory is 4GB, then in bytes, the total memory is on the order of 10. 9 The above is significantly different from the order of magnitude of the number of nodes and edges. Using a third positive parameter allows the number of nodes and edges to be directly mapped to data related to the order of magnitude of memory.

[0054] Wherein, the memory to be occupied = (number of nodes * A + number of edges * B) * C, where A is the first positive parameter, B is the second positive parameter, and C is the third positive parameter.

[0055] The purpose of all the positive parameters is to map the order of magnitude of the number of nodes and edges to the order of magnitude of memory, so as to directly obtain the memory to be occupied. In addition, the settings of the three positive parameters are set according to different use cases. Under the infoMap algorithm, the recommended value for A is 10-20, the recommended value for B is 5-15, and the recommended value for C is 20-50.

[0056] Step S203: Based on the current remaining memory and the memory to be occupied, divide the N images to be archived into M fragment sets.

[0057] In this application, the current remaining memory can refer to the amount of memory remaining in the processor of the device on which this method is running. The amount of memory remaining can be obtained through the device's own monitoring. Of course, in order to avoid the current remaining memory being occupied, it is also necessary to work with the device to reserve the current remaining memory in advance until the current archiving is completed, so as to ensure the availability of the above method.

[0058] During this process, it's necessary to consider that in a single batch processing, all remaining memory might be fully utilized, potentially causing the device to operate at full capacity and impacting its lifespan. The remaining memory refers to the amount of memory available for a single batch processing of images to be archived. If the remaining memory exceeds this amount, the N images to be archived need to be divided into different batches, processed sequentially using the remaining memory. If the remaining memory does not exceed this amount, all N images can be processed directly. Here, M is a positive integer, meaning that dividing the N images to be archived results in at least one fragment set.

[0059] The batch size can be obtained by comparing the memory to be used with the current remaining memory. For example, if the memory to be used is 1GB and the current remaining memory is 200MB, the ratio is 5. This means that N images to be archived are divided into 5 sets of fragments, each fragment set occupies 200MB of memory, and the remaining memory is used for processing sequentially. Of course, the current remaining memory here is not the only memory available for computer processing; it is essentially redundant. The memory occupied by each fragment set may be slightly larger or smaller than the current remaining memory, and the current remaining memory can still be used for processing.

[0060] Optionally, based on the current remaining memory and the memory to be used, the N images to be archived are divided into M fragment sets, including:

[0061] Based on the current remaining memory and the memory to be used, determine the minimum number of batches required to process N images to be archived in batches.

[0062] The N images to be archived are divided into M fragment sets based on the minimum batch size.

[0063] The memory to be used is the memory that needs to be consumed. By comparing the memory to be used with the current remaining memory, the number of batch processing steps required can be obtained. This number can be understood as the minimum number of batches. Based on this minimum number of batches, the N images to be archived are divided into M different sets of fragments, each of which includes at least one image to be archived.

[0064] Optionally, based on the current remaining memory and the memory to be used, determine the minimum number of batches required to process N images to be archived, including:

[0065] Based on the number of nodes and the number of edges, calculate the average number of edges for each corresponding edge. Calculate the product of the average number of edges and the second positive parameter to obtain the third product. Add the third product to the first positive parameter to obtain the sum. Multiply the sum by the third positive parameter to obtain the unit memory occupied by a single node.

[0066] Get the current remaining memory, compare the current remaining memory with the unit memory, and get the maximum number of nodes that the current remaining memory can handle;

[0067] By comparing the number of nodes with the maximum number of nodes, we can obtain the minimum number of batches required to process N images to be archived.

[0068] In one implementation of the above steps, the number of edges = number of nodes * average number of edges, the maximum number of nodes = current remaining memory / C / (A + average number of edges * B), the minimum number of batches = number of nodes / maximum number of nodes, and C * (A + average number of edges * B) expresses the unit memory occupied by a single node.

[0069] Compared to the number of times obtained by comparing the memory to be occupied with the current remaining memory, the minimum number of batches calculated using this method, by setting the average number of connected edges, can better consider the structural relationships between nodes, has more refined batching logic, can avoid the full load operation of device memory, and helps to improve device lifespan.

[0070] Step S204: Based on the M fragment sets, obtain the archiving results of N images to be archived.

[0071] Optionally, based on M fragment sets, the archived result of N images to be archived is obtained, including:

[0072] The M clustering results are merged and archived to obtain N archived images to be archived;

[0073] Cluster the M partition sets separately to obtain M clustering results, with each partition set corresponding to one clustering result.

[0074] In this application, clustering can refer to grouping at least two data sets that represent the same or similar targets into one class, meaning that all data within that class represent the same target. The data in this application is images. Images that are identical or similar can be considered to represent the same target. For example, in a face tracking scenario, the image content is a face. If the same person's face appears in two images, both images can be clustered into one class to represent the same person.

[0075] Fusion archiving can refer to merging the clustering results obtained after processing each batch. Since there may be classes in the clustering results of different batches that represent the same target, it is necessary to merge the two classes from different batches and archive them on the corresponding target.

[0076] Specifically, based on the current remaining memory, images to be archived that are similar in any fragment set are clustered into one class to obtain the clustering result of the corresponding fragment set. All fragment sets are traversed to obtain the clustering result corresponding to each fragment set. All clustering results that are similar are merged and archived to obtain at least one archived result. In any archived result, all images to be archived are represented as the same target.

[0077] Ultimately, the archived results can be used to describe the trajectory file of each target, thus achieving the purpose of tracking.

[0078] Optionally, the step of clustering the M fragment sets to obtain M clustering results includes:

[0079] In each of the aforementioned fragment sets, K images to be archived are selected as initial cluster centers;

[0080] Each piecewise set is clustered based on the similarity between the remaining images to be archived and the images to be archived corresponding to each initial cluster center point, resulting in K initial cluster sets for each piecewise set, where K is an integer greater than zero. The remaining images to be archived are the images to be archived in each piecewise set other than the images to be archived corresponding to the initial cluster center points.

[0081] Determine the set center point in each of the initial cluster sets, and use the set center point as the initial cluster center point;

[0082] Until the iteration condition is met, the clustering result of each of the fragment sets is obtained, resulting in M ​​clustering results, wherein each of the clustering results includes K cluster sets, and each of the cluster sets includes at least one image to be archived that represents the same target.

[0083] Specifically, for any given set of segments, a clustering algorithm is used to cluster the images to be archived in that set of segments, resulting in a clustering result for that set of segments. The clustering result includes K clusters, where the value of K can be an adaptively adjusted value. If the images to be archived in one set of segments correspond to 5 targets, then the value of K is 5; if the images to be archived in another set of segments correspond to 2 targets, then the value of K is 2, and so on.

[0084] The iteration condition can refer to the termination condition of cyclic clustering. Specifically, the termination condition can be that the number of iterations meets a preset number, for example, the preset number is 100. Of course, the termination condition can also be that the cluster center points change little in two adjacent iterations or that the clustering results do not change.

[0085] For example, given 100 images to be archived in a fragment set, in the first iteration, 5 images to be archived are selected as initial cluster centers, i.e., K=5. The similarity between the remaining images to be archived and the images corresponding to each initial cluster center is calculated. The images to be archived with higher similarity are clustered together with their corresponding initial cluster centers, i.e., the initial cluster set. The images to be archived corresponding to the centers in this initial cluster set are selected as the initial cluster centers. In subsequent iterations, the similarity between the remaining images to be archived and the images corresponding to each initial cluster center is calculated, and the iteration count is incremented by one. This process is repeated until the number of iterations reaches 100.

[0086] Optionally, the M clustering results are fused and archived to obtain an archived result of N images to be archived, including:

[0087] Determine the image to be archived corresponding to the set center point of each cluster set in the M clustering results, and calculate the similarity between the images to be archived corresponding to any two set center points;

[0088] The clusters corresponding to the centroids of sets with similarity greater than a similarity threshold are merged into a fusion result representing the same target;

[0089] All fusion results containing the same image to be archived are archived as archive results representing the same target, resulting in N archive results of images to be archived.

[0090] The process involves determining the image to be archived corresponding to the center point of each cluster, based on the clustered results. The similarity between any two such images is calculated, and clusters with similarity greater than a threshold are merged together. The resulting fusion image represents the same target. Finally, the fusion result is used to archive N images. Furthermore, if an image contains multiple objects, the issue of multiple objects belonging to the same target needs to be considered. Therefore, all fusion results containing the same image are archived as representing the same target.

[0091] For example, the first image to be archived contains only a face, the second image contains only a body, and the third image contains both a face and a body. In this third image, the face and body belong to the same target. If the image similarity calculation method described above is used, the first and second images might not be classified into the same category, but both could be classified with the third image. Therefore, it can be determined that the first and second images belong to the same target. Thus, if two fusion results contain the same image to be archived, it can be determined that the two fusion results belong to the same target and can be classified into the same archive, improving the accuracy of archiving.

[0092] Furthermore, when merging cluster sets, at least one image to be archived can be extracted from each cluster set as the image to be archived corresponding to the center point of the aforementioned set.

[0093] This application embodiment constructs an image connection network based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to represent the correlation between two nodes of the edge that meets the preset conditions. Based on the number of nodes in the image connection network and the number of edges in the image connection network, the memory to be occupied is determined. According to the current remaining memory and the memory to be occupied, the N images to be archived are divided into M fragment sets. Each fragment set includes at least one image to be archived. The M fragment sets are clustered respectively to obtain M clustering results. Each fragment set corresponds to one clustering result. Based on the M clustering results, the archiving result of the N images to be archived is obtained. In this method, the images to be archived are used as nodes. The complexity of the dataset of N images to be archived is described by the correlation of nodes and edges, thereby determining the required memory size. At the same time, the remaining memory guides the segmentation of the N images to be archived, so as to realize the clustering, fusion and archiving operations of different batches, and thus obtain the archiving result. The number of batches is related to the complexity of the dataset and can be applied to datasets in any situation, which has good applicability. Compared with a fixed number of batches, it can effectively improve the rational utilization of memory.

[0094] See Figure 3 This is a flowchart illustrating an image archiving method provided in Embodiment 2 of this application. Figure 3 As shown, the image archiving method includes the following steps:

[0095] Step S301: Treat each image to be archived as a node in the image connection network, detect each image to be archived, and if a target object is detected in each image to be archived, use the image information of the target object as the node information of the node corresponding to each image to be archived.

[0096] In this application, when the image to be archived contains more than one type of target, it is necessary to limit one archived image to correspond to one node, but the node information needs to be composed of image information corresponding to multiple targets.

[0097] The target object includes a first object and / or a second object. The first and second objects are used to distinguish different types of targets. For example, in a tracking scenario, in addition to face tracking, tracking can also be based on the human body. Faces and bodies represent two different types of targets. In all the above embodiments, if applied to face or body tracking scenarios, the image to be archived only contains information about one face or one body. However, in this embodiment, the image to be archived can contain information about one face, one body, or one body with a face (i.e., a complete person).

[0098] For two images to be archived that contain a first object or a second object, since the objects belong to the same type, the two images are comparable and may be related. However, if one image to be archived only contains the first object and the other image to be archived only contains the second object, then the two images are not comparable and cannot be related, so they are not considered.

[0099] Step S302: For any two nodes, if both nodes include the first object, then perform correlation calculation on the image information corresponding to the first object to obtain the correlation of the first object.

[0100] Step S303: If any two nodes include the second object, then perform correlation calculation on the image information corresponding to the second object to obtain the correlation of the second object.

[0101] Step S304: If the association of the first object satisfies the preset condition, and / or the association of the second object satisfies the preset condition, then an edge is constructed between any two nodes.

[0102] In this application, the preset conditions can be set according to the requirements. For example, under a certain preset condition, if a first object and a second object exist in one image to be archived, and a first object or a second object exists in another image to be archived, then the first object or the second object needs to be compared separately to determine the correlation. As long as one object has a correlation, it can be determined that the nodes corresponding to the two images to be archived have a connection relationship.

[0103] If a first object and a second object exist in one image to be archived, and a first object and a second object also exist in another image to be archived, then each type of object is compared one by one. If there is one object that satisfies the association, it can be determined that the nodes corresponding to the two images to be archived have a connection relationship.

[0104] Of course, under another preset condition, if a first object and a second object exist in one image to be archived, and a first object and a second object also exist in another image to be archived, then after comparing each type of object one by one, it is necessary to ensure that all of them satisfy the association condition before it can be determined that the nodes of the two images to be archived have an association relationship.

[0105] Furthermore, correlation can be expressed using similarity. The preset condition can be a threshold set for similarity. By calculating the similarity of the image information corresponding to each object class, the similarity can be obtained. Similarity calculation can use Euclidean distance, cosine similarity, etc. In this embodiment, as long as there are objects of a certain class with a similarity greater than the threshold, it can be determined that there is an edge between the corresponding nodes, reducing the clustering requirements and helping to improve the clustering effect.

[0106] Step S305: Determine the memory to be occupied based on the number of nodes in the image connection network and the number of edges in the image connection network.

[0107] Step S306: Based on the current remaining memory and the memory to be occupied, divide the N images to be archived into M fragment sets.

[0108] Step S307: Based on the M fragment sets, obtain the archiving results of N images to be archived.

[0109] The contents of steps S305 to S307 are the same as those of steps S202 to S204 above. For details, please refer to the descriptions of steps S202 to S204, which will not be repeated here.

[0110] This paper addresses the case where there is more than one type of object in the image to be archived. In this case, the construction of nodes and node information, as well as the method of connecting edges, need to be specially explained. In actual face and body tracking scenarios, this provides a more accurate processing procedure, which helps to improve the applicability of the method in this application. Furthermore, the archiving method in this case can be further optimized to achieve the final archiving purpose.

[0111] Corresponding to the image archiving method in the above embodiments, Figure 4 A structural block diagram of the image archiving device provided in Embodiment 3 of this application is shown. This image archiving device is used to connect a server-side computer device to a client, database, etc., to obtain relevant data. For example, the client is an image acquisition device capable of sending acquired real-time images to the server for archiving processing. The server running the image archiving device can be applied to scenarios such as target tracking and localization in images. This process requires image processing methods such as image recognition and classification, where AI can be used to implement specific image processing. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0112] See Figure 4 The image archiving device includes:

[0113] The image connection network construction module 41 is used to construct an image connection network based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to characterize the correlation between two nodes of the edge, which satisfies a preset condition. Here, N is an integer greater than one.

[0114] The memory determination module 42 is used to determine the memory to be occupied based on the number of nodes in the image connection network and the number of edges in the image connection network.

[0115] The sharding processing module 43 is used to divide N images to be archived into M shard sets according to the current remaining memory and the memory to be occupied. Each shard set includes at least one image to be archived, and M is a positive integer.

[0116] The fragment archiving module 44 is used to obtain the archiving result of N images to be archived based on M fragment sets.

[0117] Optionally, the memory determination module 42 includes:

[0118] The parameter acquisition unit is used to acquire a first positive parameter and a second positive parameter that are on the same order of magnitude as the number of adjustment nodes and the number of edges, and to acquire a third positive parameter that is on the same order of magnitude as the number of memory.

[0119] The memory determination unit is used to determine the quantity to be calculated based on the first positive parameter, the second positive parameter, the number of nodes, and the number of edges, and to determine the memory to be occupied based on the third positive parameter and the quantity to be calculated.

[0120] Optionally, the sharding processing module 43 includes:

[0121] The batch number determination unit is used to determine the minimum batch number required to process N images to be archived in batches based on the current remaining memory and the memory to be used.

[0122] The fragmentation processing unit is used to fragment N images to be archived according to the minimum batch number, resulting in M ​​fragment sets.

[0123] Optionally, the network construction module 41 includes:

[0124] The node information determination unit is used to treat each image to be archived as a node in the image connection network, detect each image to be archived, and if a target object is detected in each image to be archived, the image information of the target object is used as the node information of the node corresponding to each image to be archived. The target object includes a first object and / or a second object.

[0125] The first correlation determination unit is used to perform correlation calculation on the image information corresponding to the first object for any two nodes if both nodes include the first object, so as to obtain the correlation of the first object.

[0126] The second correlation determination unit is used to perform correlation calculation on the image information corresponding to the second object if any two nodes include the second object, and obtain the correlation of the second object.

[0127] An edge construction unit is used to construct an edge between any two nodes if the association of a first object satisfies a preset condition, and / or the association of a second object satisfies a preset condition.

[0128] Optionally, the fragmented archiving module 44 includes:

[0129] The partitioned clustering unit is used to cluster M partitioned sets respectively, resulting in M ​​clustering results, with each partitioned set corresponding to one clustering result.

[0130] The fusion archiving unit is used to fuse and archive M clustering results to obtain N archived images.

[0131] Optionally, the piecewise clustering unit includes:

[0132] The initial center point determination sub-unit is used to select K images to be archived as initial cluster centers in each fragment set;

[0133] The clustering subunit is used to cluster each of the fragment sets based on the similarity between the remaining images to be archived and the images to be archived corresponding to each initial cluster center point, to obtain K initial cluster sets for each fragment set, where K is an integer greater than zero, and the remaining images to be archived are the images to be archived in each fragment set other than the images to be archived corresponding to the initial cluster center points.

[0134] The set center point determination sub-unit is used to determine the set center point in each initial cluster set, and the set center point is used as the initial cluster center point;

[0135] The clustering results determine sub-units, which are used until the iteration conditions are met to obtain the clustering results of each piece set, so as to obtain M clustering results, wherein each clustering result includes K cluster sets, and each cluster set includes at least one image to be archived that represents the same target.

[0136] Optionally, the merged archiving unit includes:

[0137] The similarity calculation subunit is used to determine the image to be archived corresponding to the set center point of each cluster set in the M clustering results, and to calculate the similarity between the images to be archived corresponding to any two set center points.

[0138] The fusion subunit is used to merge the clusters corresponding to the center points of sets with similarity greater than a similarity threshold into a fusion result representing the same target;

[0139] The archiving subunit is used to archive all fusion results containing the same image to be archived into an archive result representing the same target, resulting in an archive result of N images to be archived.

[0140] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0141] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of this application. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above-described image archiving method embodiments.

[0142] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0143] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0144] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0145] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0146] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image archiving method characterized by, The image archiving method includes: An image connection network is constructed based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to characterize the correlation between two nodes of the edge, which satisfies a preset condition. Here, N is an integer greater than one. The memory to be occupied is determined based on the number of nodes in the image connection network and the number of edges in the image connection network. Based on the current remaining memory and the memory to be occupied, the N images to be archived are divided into M fragment sets, each fragment set including at least one image to be archived, where M is a positive integer; Based on the M fragment sets, the archiving results of the N images to be archived are obtained; The step of determining the memory to be occupied based on the number of nodes and the number of edges in the image connectivity network includes: Obtain a first positive parameter and a second positive parameter that adjust the number of nodes and the number of edges to be on the same order of magnitude, and obtain a third positive parameter that adjusts the number of nodes and the number of edges to be on the same order of magnitude as the amount of memory; Based on the first positive parameter, the second positive parameter, the number of nodes, and the number of edges, the quantity to be calculated is obtained; Based on the third positive parameter and the quantity to be calculated, the memory to be occupied is obtained. All the positive parameters correspond the order of magnitude of the number of nodes and the number of edges to the order of magnitude of the memory, so as to directly obtain the memory to be occupied.

2. The image filing method according to claim 1, characterized by, The step of dividing the N images to be archived into M fragment sets based on the current remaining memory and the memory to be occupied includes: Based on the current remaining memory and the memory to be used, determine the minimum number of batches required to process the N images to be archived in batches; The N images to be archived are divided into M fragment sets according to the minimum batch number.

3. The image archiving method according to any one of claims 1 to 2, characterized in that, The construction of the image connection network based on N images to be archived includes: Each of the images to be archived is treated as a node in the image connection network. Each of the images to be archived is detected. If a target object is detected in each of the images to be archived, the image information of the target object is used as the node information of the node corresponding to each image to be archived. The target object includes a first object and / or a second object. For any two nodes, if both nodes include the first object, then the image information corresponding to the first object is used to perform correlation calculation to obtain the correlation of the first object. If any two nodes include the second object, then the image information corresponding to the second object is used to calculate the correlation to obtain the correlation of the second object; If the association of the first object satisfies the preset condition, and / or the association of the second object satisfies the preset condition, then an edge is constructed between any two nodes.

4. The image filing method according to claim 1, characterized by, The archiving result of the N images to be archived based on the M fragment sets includes: Clustering is performed on the M partition sets respectively to obtain M clustering results, with each partition set corresponding to one clustering result; The M clustering results are fused and archived to obtain the archived results of the N images to be archived.

5. The image filing method according to claim 4, characterized by, The process of clustering the M partitioned sets to obtain M clustering results includes: In each of the aforementioned fragment sets, K images to be archived are selected as initial cluster centers; Each piecewise set is clustered based on the similarity between the remaining images to be archived and the images to be archived corresponding to each initial cluster center point, resulting in K initial cluster sets for each piecewise set, where K is an integer greater than zero. The remaining images to be archived are the images to be archived in each piecewise set other than the images to be archived corresponding to the initial cluster center points. Determine the set center point in each of the initial cluster sets, and use the set center point as the initial cluster center point; Until the iteration condition is met, the clustering result of each of the fragment sets is obtained, resulting in M ​​clustering results, wherein each of the clustering results includes K cluster sets, and each of the cluster sets includes at least one image to be archived that represents the same target.

6. The image filing method according to claim 5, characterized by, The process of fusing and archiving the M clustering results to obtain the archived results of the N images to be archived includes: Determine the image to be archived corresponding to the set center point of each cluster set in the M clustering results, and calculate the similarity between the images to be archived corresponding to any two set center points; The clusters corresponding to the centroids of sets with similarity greater than a similarity threshold are merged into a fusion result representing the same target; The fusion results containing the same image to be archived are archived as archive results representing the same target, thus obtaining the archive results of the N images to be archived.

7. An image archiving apparatus characterized by comprising: The image archiving device includes: A network construction module is used to construct an image connection network based on N images to be archived. The nodes of the image connection network are the images to be archived, and the edges of the image connection network are used to characterize the correlation between two nodes of the edge, which satisfies a preset condition. Here, N is an integer greater than one. The memory determination module is used to determine the memory to be occupied based on the number of nodes in the image connection network and the number of edges in the image connection network. The sharding processing module is used to divide the N images to be archived into M shard sets according to the current remaining memory and the memory to be occupied, where each shard set includes at least one image to be archived, and M is a positive integer; The fragment archiving module is used to obtain the archiving results of the N images to be archived based on the M fragment sets; The memory determination module includes: The parameter acquisition unit is used to acquire a first positive parameter and a second positive parameter that adjust the number of nodes and the number of edges to be on the same order of magnitude, and to acquire a third positive parameter that adjusts the number of nodes and the number of edges to be on the same order of magnitude as the number of memory. The memory determination unit is used to obtain the quantity to be calculated based on the first positive parameter, the second positive parameter, the number of nodes, and the number of edges; Based on the third positive parameter and the quantity to be calculated, the memory to be occupied is obtained. All the positive parameters correspond the order of magnitude of the number of nodes and the number of edges to the order of magnitude of the memory, so as to directly obtain the memory to be occupied.

8. A computer device, comprising: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image archiving method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by a processor, it implements the image archiving method as described in any one of claims 1 to 6.

Images