A method, apparatus and electronic device for multi-bin merging

Abstract

The application relates to the field of image processing, in particular to a multi-file merging method and device and electronic equipment, which are used for solving the problem of one person having multiple files in the existing person file. The method comprises the following steps: acquiring M person track sequences generated based on a first person file, calculating track distance values between each person track sequence and a reference track sequence, obtaining a plurality of track distance values, selecting a plurality of track distance values meeting a preset condition from the plurality of track distance values, and determining vehicle identifiers of the reference track sequences corresponding to the plurality of track distance values. Further, if there is another person file, for example, a second person file, associated with the vehicle identifier of the non-first person file, the method further comprises the following steps: calculating a file similarity value between the first person file and the second person file, and in response to the file similarity value being greater than a preset threshold, merging the first person file and the second person file. Based on the above method, the multi-file merging can be performed to optimize the person file.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and electronic device for merging multiple files. Background Technology

[0002] To meet the growing demand for security infrastructure in my country, intelligent surveillance equipment (hereinafter referred to as "vehicle cards") is being deployed at various intersections in cities. Currently, vehicle cards can capture facial images through vehicle windows. These images are then clustered using facial feature values ​​to create portrait profiles, which can assist in related security operations.

[0003] However, due to environmental factors such as weather, obstruction, and reflection from car windows, the clarity of facial images captured by car windows is often poor. Clustering based solely on the similarity of facial images captured by car windows can result in multiple images of the same target being clustered into multiple portrait profiles. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for merging multiple files to reduce the rate of multiple files in portrait archives.

[0005] Firstly, this application provides a method for merging multiple files, the method comprising:

[0006] Obtain M sequences of facial trajectory data generated based on the first facial profile; where M is an integer greater than 0.

[0007] For the M human image trajectory sequences, the trajectory distance value between each human image trajectory sequence and the reference trajectory sequence is calculated to obtain several trajectory distance values; wherein, the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period;

[0008] Select multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determine the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values;

[0009] If the vehicle identification is associated with a second facial image file, then calculate the file similarity value between the first facial image file and the second facial image file;

[0010] In response to the file similarity value being greater than a preset threshold, the first portrait file and the second portrait file are merged.

[0011] In one possible implementation, obtaining the M portrait trajectory sequences generated based on the first portrait file includes:

[0012] Obtain a first portrait file of the first target object; wherein the first portrait file includes at least: multiple facial images of the first target object captured by car windows;

[0013] According to the image acquisition time, multiple face images of car windows in the first portrait file are sorted to obtain the first portrait trajectory sequence;

[0014] In the first trajectory sequence, duplicate images of faces in the car window that are continuously in the same spatial position are deduplicated to obtain the second portrait trajectory sequence.

[0015] Based on the preset M segmentation time periods, the face images in the car window of the second portrait trajectory sequence are divided to obtain M portrait trajectory sequences.

[0016] In one possible implementation, calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes:

[0017] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0018] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0019] In response to the fact that the sequence length of the reference trajectory sequence is greater than the sequence length of the single human portrait trajectory sequence, for each of the m elements in the single human portrait trajectory sequence, the trajectory distance value is calculated by sequentially multiplying the spatial position of each element with the spatial position of each of the n elements in the reference trajectory sequence, resulting in n-m+1 trajectory distance values; where m and n are integers greater than 0, and m is less than n;

[0020] From the n-m+1 trajectory distance values, the smallest trajectory distance value is selected as the trajectory distance value between the single human portrait trajectory sequence and the reference trajectory sequence.

[0021] In one possible implementation, for each of the m elements in the single portrait trajectory sequence, the trajectory distance values ​​are obtained by calculating the sum of the spatial positions of each element and the spatial positions of individual elements in the n elements of the reference trajectory sequence, in order, resulting in n-m+1 trajectory distance values, including:

[0022] For a single trajectory distance value among the n-m+1 trajectory distance values, the following method is used for calculation:

[0023] Determine the first row matrix composed of the horizontal arrangement of the spatial positions of m elements in the single human portrait trajectory sequence;

[0024] Determine the first column matrix composed of the vertical arrangement of the spatial positions of n elements in the single reference trajectory sequence;

[0025] Determine the second column matrix by selecting m elements sequentially from the first column matrix; wherein the number of the second column matrices is n-m+1.

[0026] The product of the first row matrix and the second column matrix is ​​calculated as the distance value of the single trajectory.

[0027] In one possible implementation, calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes:

[0028] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0029] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0030] In response to the fact that the sequence length of the reference trajectory sequence is less than the sequence length of the single human portrait trajectory sequence, the reference trajectory sequence is padded based on the first or last element of the n elements in the reference trajectory sequence; where n is an integer greater than 0.

[0031] The trajectory distance between each portrait sequence and the reference trajectory sequence is calculated until the sequence length of the reference trajectory sequence is equal to or greater than the sequence length of the single portrait trajectory sequence.

[0032] In one possible implementation, calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes:

[0033] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0034] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0035] In response to the fact that the sequence length of the reference trajectory sequence is equal to the sequence length of the single human image trajectory sequence, the m elements of the single human image trajectory sequence are multiplied by the m elements of the reference trajectory sequence in the order of element arrangement to obtain m products;

[0036] Calculate the sum of the m products, and use the sum as the trajectory distance between the single human image trajectory sequence and the reference trajectory sequence.

[0037] In one possible implementation, calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes:

[0038] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0039] If at least one vehicle trajectory sequence does not have a reference trajectory sequence that matches the acquisition time period of the single human image trajectory sequence, then it is determined that the single human image trajectory sequence does not have a trajectory distance value, or a specified value is used as the trajectory distance value corresponding to the single human image trajectory sequence; wherein, the specified value includes at least positive infinity.

[0040] In one possible implementation, selecting multiple trajectory distance values ​​that satisfy preset conditions from the plurality of trajectory distance values ​​includes:

[0041] Arrange the trajectory distance values ​​in ascending order to obtain the sorting order of the several trajectory distance values;

[0042] Based on the arrangement order, multiple trajectory distance values ​​are selected from the plurality of trajectory distance values ​​in a selection manner from front to back.

[0043] In one possible implementation, after determining the vehicle identifier corresponding to each of the plurality of trajectory distance values, the method further includes:

[0044] Associate the first portrait file with the vehicle identification.

[0045] In one possible implementation, before calculating the file similarity value between the first and second image files if the vehicle identifier is associated with a second image file, the method further includes:

[0046] If the vehicle identifier is associated with multiple portrait files, then the file similarity value between the first portrait file and the multiple portrait files is calculated in pairs to obtain multiple file similarity values;

[0047] From the multiple file similarity values, file similarity values ​​greater than a preset threshold are selected as target similarity values, and the portrait files corresponding to the target similarity values ​​are merged.

[0048] In one possible implementation, calculating the file similarity value between the first portrait file and the second portrait file includes:

[0049] Based on the image feature values ​​of each element in the first portrait file, calculate the first file feature value of the first portrait file;

[0050] Based on the image feature values ​​of each element in the second portrait file, calculate the second file feature value of the second portrait file;

[0051] Calculate the similarity value between the first archive feature value and the second archive feature value, and use it as the archive similarity value between the first portrait archive and the second portrait archive.

[0052] In one possible implementation, merging the first portrait file and the second portrait file includes:

[0053] Map the first portrait file to the second portrait file to obtain a first mapping relationship, and save the first mapping relationship to the second portrait file; or

[0054] The second portrait file is mapped onto the first portrait file to obtain a second mapping relationship, and the second mapping relationship is saved into the first portrait file.

[0055] In summary:

[0056] This application proposes a method for merging multiple image files to reduce the rate of multiple image files. The method involves acquiring M image trajectory sequences generated based on a first image file, calculating the trajectory distance value between each image trajectory sequence and a reference trajectory sequence to obtain several trajectory distance values, then selecting multiple trajectory distance values ​​that meet preset conditions from these values, and determining the vehicle identifier corresponding to each of these multiple trajectory distance values ​​in the reference trajectory sequence. Further, if a vehicle identifier is associated with another image file not in the first image file, such as a second image file, the file similarity value between the first and second image files is calculated. In response to the file similarity value being greater than a preset threshold, the first and second image files are merged.

[0057] Where M is an integer greater than 0, and the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period.

[0058] In this embodiment of the application, by introducing a reference trajectory sequence, namely a vehicle trajectory sequence that matches the collection time period with the corresponding human image trajectory sequence, the multi-file merging based on the human image trajectory sequence and the reference trajectory sequence is realized, so as to solve the problem of optimizing the multi-file human image archive and improve the accuracy of multi-file merging.

[0059] Secondly, this application provides an apparatus for merging multiple files, the apparatus comprising:

[0060] The acquisition module acquires M sequences of facial trajectory data generated based on the first facial profile; where M is an integer greater than 0.

[0061] The first calculation module calculates the trajectory distance value between each of the M human image trajectory sequences and the reference trajectory sequence to obtain several trajectory distance values; wherein, the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period;

[0062] The determination module selects multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determines the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values;

[0063] The second calculation module calculates the file similarity value between the first image file and the second image file if the vehicle identifier is associated with a second image file.

[0064] The merging module merges the first portrait file and the second portrait file in response to the file similarity value being greater than a preset threshold.

[0065] In one possible implementation, the acquisition module, which acquires M sequences of facial trajectories generated based on the first facial profile, is specifically used for:

[0066] Obtain a first portrait file of the first target object; wherein the first portrait file includes at least: multiple facial images of the first target object captured by car windows;

[0067] According to the image acquisition time, multiple face images of car windows in the first portrait file are sorted to obtain the first portrait trajectory sequence;

[0068] In the first trajectory sequence, duplicate images of faces in the car window that are continuously in the same spatial position are deduplicated to obtain the second portrait trajectory sequence.

[0069] Based on the preset M segmentation time periods, the face images in the car window of the second portrait trajectory sequence are divided to obtain M portrait trajectory sequences.

[0070] In one possible implementation, the first calculation module, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0071] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0072] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0073] In response to the fact that the sequence length of the reference trajectory sequence is greater than the sequence length of the single human portrait trajectory sequence, for each of the m elements in the single human portrait trajectory sequence, the trajectory distance value is calculated by sequentially multiplying the spatial position of each element with the spatial position of each of the n elements in the reference trajectory sequence, resulting in n-m+1 trajectory distance values; where m and n are integers greater than 0, and m is less than n;

[0074] From the n-m+1 trajectory distance values, the smallest trajectory distance value is selected as the trajectory distance value between the single human portrait trajectory sequence and the reference trajectory sequence.

[0075] In one possible implementation, for each of the m elements in the single portrait trajectory sequence, the trajectory distance values ​​are obtained by calculating the sum of the spatial positions of each element and the spatial positions of individual elements in the n elements of the reference trajectory sequence, in order, resulting in n-m+1 trajectory distance values. The first calculation module is further configured to:

[0076] For a single trajectory distance value among the n-m+1 trajectory distance values, the following method is used for calculation:

[0077] Determine the first row matrix composed of the horizontal arrangement of the spatial positions of m elements in the single human portrait trajectory sequence;

[0078] Determine the first column matrix composed of the vertical arrangement of the spatial positions of n elements in the single reference trajectory sequence;

[0079] Determine the second column matrix by selecting m elements sequentially from the first column matrix; wherein the number of the second column matrices is n-m+1.

[0080] The product of the first row matrix and the second column matrix is ​​calculated as the distance value of the single trajectory.

[0081] In one possible implementation, the first calculation module, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0082] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0083] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0084] In response to the fact that the sequence length of the reference trajectory sequence is less than the sequence length of the single human portrait trajectory sequence, the reference trajectory sequence is padded based on the first or last element of the n elements in the reference trajectory sequence; where n is an integer greater than 0.

[0085] The trajectory distance between each portrait sequence and the reference trajectory sequence is calculated until the sequence length of the reference trajectory sequence is equal to or greater than the sequence length of the single portrait trajectory sequence.

[0086] In one possible implementation, the first calculation module, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0087] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0088] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0089] In response to the fact that the sequence length of the reference trajectory sequence is equal to the sequence length of the single human image trajectory sequence, the m elements of the single human image trajectory sequence are multiplied by the m elements of the reference trajectory sequence in the order of element arrangement to obtain m products;

[0090] Calculate the sum of the m products, and use the sum as the trajectory distance between the single human image trajectory sequence and the reference trajectory sequence.

[0091] In one possible implementation, the first calculation module, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0092] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0093] If at least one vehicle trajectory sequence does not have a reference trajectory sequence that matches the acquisition time period of the single human image trajectory sequence, then it is determined that the single human image trajectory sequence does not have a trajectory distance value, or a specified value is used as the trajectory distance value corresponding to the single human image trajectory sequence; wherein, the specified value includes at least positive infinity.

[0094] In one possible implementation, the step of selecting multiple trajectory distance values ​​that satisfy preset conditions from the plurality of trajectory distance values, the determining module, is specifically used for:

[0095] Arrange the trajectory distance values ​​in ascending order to obtain the sorting order of the several trajectory distance values;

[0096] Based on the arrangement order, multiple trajectory distance values ​​are selected from the plurality of trajectory distance values ​​in a selection manner from front to back.

[0097] In one possible implementation, after determining the vehicle identifiers corresponding to each of the plurality of trajectory distance values, the determining module is further configured to:

[0098] Associate the first portrait file with the vehicle identification.

[0099] In one possible implementation, before calculating the file similarity value between the first and second image files if the vehicle identifier is associated with a second image file, the second calculation module is further configured to:

[0100] If the vehicle identifier is associated with multiple portrait files, then the file similarity value between the first portrait file and the multiple portrait files is calculated in pairs to obtain multiple file similarity values;

[0101] From the multiple file similarity values, file similarity values ​​greater than a preset threshold are selected as target similarity values, and the portrait files corresponding to the target similarity values ​​are merged.

[0102] In one possible implementation, the second calculation module, in calculating the file similarity value between the first portrait file and the second portrait file, is specifically used for:

[0103] Based on the image feature values ​​of each element in the first portrait file, calculate the first file feature value of the first portrait file;

[0104] Based on the image feature values ​​of each element in the second portrait file, calculate the second file feature value of the second portrait file;

[0105] Calculate the similarity value between the first archive feature value and the second archive feature value, and use it as the archive similarity value between the first portrait archive and the second portrait archive.

[0106] In one possible implementation, the merging module, which merges the first portrait file and the second portrait file, is specifically used for:

[0107] Map the first portrait file to the second portrait file to obtain a first mapping relationship, and save the first mapping relationship to the second portrait file; or

[0108] The second portrait file is mapped onto the first portrait file to obtain a second mapping relationship, and the second mapping relationship is saved into the first portrait file.

[0109] Thirdly, this application provides an electronic device, the electronic device comprising:

[0110] Memory, used to store computer programs;

[0111] When the processor executes the computer program stored in the memory, it implements the above-described method steps for merging multiple files.

[0112] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps for merging multiple files.

[0113] For the various aspects of the second to fourth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, which will not be repeated here. Attached Figure Description

[0114] Figure 1 A schematic diagram illustrating one possible application scenario provided by this application;

[0115] Figure 2 A flowchart of a method for merging multiple files provided in this application;

[0116] Figure 3 A flowchart illustrating a method for generating a human image trajectory sequence provided in this application;

[0117] Figure 4 A schematic diagram for calculating file similarity values ​​provided in this application;

[0118] Figure 5 A schematic diagram of a multi-stage merging device provided in this application;

[0119] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0120] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0121] First, some terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0122] (1) One person, multiple files: Multiple facial images of the same target subject captured by the vehicle window are clustered into multiple portrait files.

[0123] (2) Image Feature Values: For example, vehicle face image data captured by front-end capture devices such as cameras. The data includes the capture camera number, GPS (Global Positioning System) coordinates, captured image, capture time, etc. Optionally, these data are stored in the back-end server. The back-end server uses a parsing algorithm to analyze the features of each dimension in the image. The parsing result is an X-dimensional vector, which is the image feature value of the image. Of course, the image feature value can also be stored in the database along with other data.

[0124] (3) Centroid of the archive mean: The sum of the image feature values ​​of each image in the archive divided by the number of images in the archive, that is, the average value of the image feature values.

[0125] It should be noted that this solution can be applied to various application scenarios such as security management, intelligent monitoring, smart cities, and mobile intelligence, including merging multiple files for one person. This solution is also suitable for tasks requiring improved and optimized accuracy in merging multiple files.

[0126] The implementation entities of this solution can be computing devices such as video surveillance cameras, computing terminals, and remote servers. By deploying on relevant computing devices and utilizing the similarity between facial image trajectory sequences and vehicle trajectory sequences, the problem of multiple profiles for one person in facial image files can be optimized, thereby improving the accuracy of merging multiple profiles. Of course, this is only an illustrative example of the entities on which this solution can be applied, and no specific limitations are imposed on them.

[0127] The design concept of the multi-file merging method provided in the embodiments of this application will be briefly introduced below.

[0128] With the increasing demands of national defense, more and more devices are capable of clustering data. For example, for facial image data from car windows, identical or similar faces can be grouped together to create a portrait archive.

[0129] In smart city scenarios, intelligent monitoring devices (hereinafter referred to as "vehicle cards") cover various intersections in the city. These vehicle cards capture facial images of people seen through car windows, and then cluster these images to create individual profiles. However, due to objective environmental factors such as weather, obstructions, and window reflections, the clarity of these facial images may be poor. Therefore, if clustering is based solely on the similarity of the facial images captured through car windows, multiple images of a single person captured through car windows may be clustered into multiple individual profiles.

[0130] It is evident that the relevant technologies result in a high rate of multiple files in the facial image archive.

[0131] To reduce the rate of multiple image profiles, this application proposes a method for merging multiple profiles. In this method, M image trajectory sequences generated based on a first image profile are obtained, and the trajectory distance value between each image trajectory sequence and a reference trajectory sequence is calculated to obtain several trajectory distance values. Then, several trajectory distance values ​​that meet preset conditions are selected from these several trajectory distance values, and the vehicle identifier corresponding to each of these multiple trajectory distance values ​​in the reference trajectory sequence is determined. Further, if there is a vehicle identifier associated with other image profiles not in the first image profile, such as a second image profile, the profile similarity value between the first image profile and the second image profile is calculated. In response to the profile similarity value being greater than a preset threshold, the first image profile and the second image profile are merged.

[0132] Where M is an integer greater than 0, and the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period.

[0133] In this embodiment of the application, by introducing a reference trajectory sequence, namely a vehicle trajectory sequence that matches the collection time period with the corresponding human image trajectory sequence, the multi-file merging based on the human image trajectory sequence and the reference trajectory sequence is realized, so as to solve the problem of optimizing the multi-file human image archive and improve the accuracy of multi-file merging.

[0134] The multi-file merging method provided in this application embodiment can be applied to... Figure 1 The implementation environment shown may include at least camera nodes, operation nodes, management nodes, computing nodes, and storage nodes.

[0135] exist Figure 1 In this system, camera nodes include spherical, cylindrical, and other types of cameras. These camera nodes can be deployed on urban roads or intersections to capture images or record videos. In this embodiment, the human image trajectory sequence consists of face images from vehicle windows, and the vehicle trajectory sequence consists of vehicle images. The face images from vehicle windows or vehicle images include, but are not limited to: images captured by the camera nodes, images cropped from the captured images, and images cropped from the recorded video. Each image includes at least one object, such as a vehicle or a face in a vehicle window.

[0136] exist Figure 1 In this context, the operation node is used to interact with the user, enabling the user to deploy, configure, and manage multi-file merging tasks of portrait archives.

[0137] exist Figure 1 In this context, the management node is used to obtain images and videos from the camera nodes; for example, see [link to relevant documentation]. Figure 1 The camera nodes upload images and videos to the cloud, and the management nodes retrieve these images and videos from the cloud. The management nodes are also used to manage compute and storage nodes in conjunction with multiple merged tasks. During the management process, the management nodes forward images and videos to the compute nodes.

[0138] exist Figure 1 In this system, computing nodes are used to complete the computational tasks involved in multi-level merging based on the received images and videos, thereby accelerating computation.

[0139] exist Figure 1 In this context, storage nodes are used to manage and store images captured by camera nodes, recorded videos, and various mapping relationships generated through a multi-file merging process, in order to trace the source.

[0140] It should be noted that the management node, compute node, and storage node mentioned above are different devices, or any two or three of the management node, compute node, and storage node can be integrated into the same device. The aforementioned operation nodes are not mandatory, and this solution does not impose specific limitations on them.

[0141] The multi-file merging method provided in the embodiments of this application will be described in detail below. (Reference) Figure 2 The method includes steps 21-25, as detailed below.

[0142] Step 21: Obtain M sequences of facial trajectories generated based on the first facial profile;

[0143] If the first portrait file is used to identify the first target object, then the first portrait file contains at least multiple facial images of the person captured from the car window for the first target object.

[0144] In this embodiment, based on the acquisition time of the aforementioned multiple car window face images, a first image trajectory sequence of the first image file is constructed according to the time sequence. Then, based on the spatial location (e.g., latitude and longitude coordinates) of the aforementioned multiple car window face images, deduplication processing is performed on car window face images in the first image trajectory sequence that are in the same (or similar) spatial location, according to spatial logic. Optionally, deduplication processing is also performed on car window face images acquired at the same or similar times to obtain a second image trajectory sequence. Finally, through a preset M (M is an integer greater than 0) segmentation time periods, the car window face images in the second image trajectory sequence are divided, thereby obtaining M image trajectory sequences generated based on the first image file.

[0145] For example, see Figure 3 The diagram shows a flowchart of generating M human image trajectory sequences provided in this application, including steps 301-304, as detailed below.

[0146] Step 301: Obtain the first portrait file of the first target object;

[0147] In some implementations, the first portrait profile is, for example, a portrait profile formed by clustering facial image data from vehicle windows. Specifically, during the generation of the portrait profile data, to save transmission, computation, and / or storage resources, captured data from vehicles in central or key areas of the city (such as shopping malls, hospitals, schools, etc.) can be filtered. Then, facial clustering is performed on the captured images of faces in the vehicle windows from the filtered data to form the portrait profile data; that is, the facial images of faces in the vehicle windows in the portrait profile data are images captured on roads or intersections in central or key areas of the city.

[0148] In some implementations, the first facial profile is a clustered facial profile data, which may include: profile number, facial image captured by a vehicle window, facial image features, acquisition time (or capture time), latitude and longitude coordinates, etc. Optionally, the facial profile data may be stored in a table on a storage node or in a backend database.

[0149] For example, the aforementioned face image features from the car window can be an X-dimensional matrix parsed from the face image from the car window using a computer vision (CV) algorithm. Furthermore, to facilitate data storage, the face image feature values ​​from the car window can be encoded. Correspondingly, when using the face image feature values ​​from the car window, decoding is performed according to the encoding rules to reconstruct the face image feature values ​​from the car window.

[0150] Step 302: Sort the multiple face images from the car window in the first portrait file according to the image acquisition time to obtain the first portrait trajectory sequence;

[0151] For example, the acquisition time of each car window face image in the first portrait file is obtained, and the car window face images in the first portrait file are sorted according to the order of acquisition time. Then, the sorted car window face image sequence is used as the first portrait trajectory sequence.

[0152] The aforementioned first human portrait trajectory sequence can be used to characterize the temporal trajectory of the first target object.

[0153] Step 303: In the first trajectory sequence, deduplication is performed on the car window face images that are continuously in the same spatial position to obtain the second portrait trajectory sequence;

[0154] In practical applications, multiple near-identical facial images of the same target object can be captured (captured) from vehicle windows within a relatively short period of time. After aggregation, these images are grouped into a single image file. Therefore, it is necessary to perform deduplication processing on these near-identical facial images generated in this application scenario.

[0155] In this embodiment of the application, the deduplication process for the first trajectory sequence is to retain only the first face image of the car window that is continuously in the same spatial location (e.g., latitude and longitude) to obtain the second portrait trajectory sequence.

[0156] For example, file A is a first trajectory sequence composed of 7 face images from car windows arranged in order. Each face image from a car window is treated as a sub-file of image file A. File A contains 7 sub-files, namely sub-file A1, sub-file A2, sub-file A3, sub-file A4, sub-file A5, sub-file A6, and sub-file A7. The sub-files that have been deduplicated and retained in the second image trajectory sequence are identified by labels. For example, only the sub-file with label=1 is retained, as shown in Table 1 below.

[0157]

[0158] Table 1

[0159] The first trajectory sequence can be represented as: sub-files A1, A2, A3, A4, A5, A6, and A7. Sub-files at the same latitude and longitude are: sub-files A1 and A2; sub-files A4 and A5; and sub-files A6 and A7. The first sub-file at the same latitude and longitude is retained, namely: sub-files A1, A4, and A6; in other words, sub-files A2, A5, and A7 are removed. After deduplication, the second trajectory sequence can be represented as: sub-files A1, A3, A4, and A6.

[0160] It should be noted that the above deduplication process is only one possible implementation for the same spatial location. If a distance threshold is set for the spatial location, then any two window face images whose spatial location is less than the distance threshold can be regarded as window face images in the same spatial location.

[0161] As an optional implementation, deduplication can also be performed on car window face images corresponding to the same acquisition time. The idea is similar to deduplication based on spatial location, and will not be repeated here. Of course, if a time difference threshold is set for the acquisition time, any two car window face images whose acquisition time difference is less than the time difference threshold can be regarded as car window face images corresponding to the same acquisition time.

[0162] Furthermore, based on the deduplication concept provided in the embodiments of this application, the deduplication operation can also be performed before generating the first portrait trajectory sequence. This solution does not impose specific limitations on the order of deduplication.

[0163] Step 304: Based on the preset M segmentation time periods, divide the face images in the car window of the second portrait trajectory sequence to obtain M portrait trajectory sequences.

[0164] Based on the acquisition time, M segmentation time periods are set, and the face images in the car window in the second portrait trajectory sequence are grouped and divided to obtain M portrait trajectory sequences.

[0165] In some implementations, the above-mentioned segmentation of time periods can be understood as converting the acquisition time into a date, such as a year, a day of the year, or an hour of the day. Based on the converted date, i.e., year, day, and hour, the face images in the car window in the second portrait dimension column are grouped and divided to obtain M portrait trajectory sequences.

[0166] For example, see Table 2 below, which shows three portrait trajectory sequences obtained based on the conversion date for portrait file A.

[0167]

[0168] Table 2

[0169] The first trajectory sequence [[104.739835, 31.489459]] is the image trajectory sequence of file A during the 10th hour of the 121st day of 2022; the second trajectory sequence [[104.725586, 31.50912], [104.741385, 31.498601]] is the image trajectory sequence of file A during the 11th hour of the 121st day of 2022; and the third trajectory sequence [[104.739835, 31.489459]] is the image trajectory sequence of file A during the 9th hour of the 122nd day of 2022.

[0170] It should be noted that steps 301-304 above are an implementation description for generating M portrait trajectory sequences for the first portrait file. Based on the same idea, the face image data of the car window or the portrait file data can be directly processed to generate portrait trajectory sequences for each portrait file data. This solution does not make specific limitations on this.

[0171] Furthermore, as a possible implementation, based on the same idea of ​​generating the human image trajectory sequence in steps 301-304 above, it is also possible to obtain the vehicle trajectory sequence corresponding to vehicle data or vehicle file data.

[0172] For example, for vehicle files generated by vehicle data clustering, multiple vehicle images in each vehicle file are sorted according to image acquisition time to obtain a first vehicle trajectory sequence corresponding to each vehicle file. Then, in the first vehicle trajectory sequence corresponding to each vehicle file, vehicle images that are consecutively located in the same spatial position or acquired at the same time are deduplicated to obtain a second vehicle trajectory sequence corresponding to each vehicle file. Finally, based on a preset M segmentation time period, the face images of car windows in the second vehicle trajectory sequence corresponding to each vehicle file are divided to obtain one or more vehicle trajectory sequences corresponding to each vehicle file.

[0173] Step 22: For the M portrait trajectory sequences, calculate the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence to obtain several trajectory distance values;

[0174] Among them, the above-mentioned reference trajectory sequence is the vehicle trajectory sequence that matches the collection period with the corresponding human image trajectory sequence.

[0175] In this embodiment, it is necessary to perform sequential calculations for M human image trajectory sequences. Taking a single human image trajectory sequence as an example, the single human image trajectory sequence is first matched sequentially with at least one (or all) vehicle trajectory sequences. Then, reference trajectory sequences matching the collection time period are selected from these vehicle trajectory sequences. Of course, the number of reference trajectory sequences selected here is not limited.

[0176] For example, the aforementioned single human image trajectory sequence is a human image trajectory sequence divided according to year, day, and hour. The aforementioned at least one (or all) vehicle trajectory sequences are one or more vehicle trajectory sequences divided according to year, day, and hour. Then the aforementioned collection period is a time period divided according to year, day, and hour.

[0177] In some implementations, the acquisition time period of a single human image sequence is determined. When a single human image trajectory sequence matches a vehicle trajectory sequence during that acquisition time period, the matched vehicle spatiotemporal trajectory is used as a reference trajectory sequence for the single human image trajectory sequence.

[0178] Furthermore, taking the selection of a single reference trajectory sequence as an example, the sequence lengths between the single portrait trajectory sequence and the single reference trajectory sequence are compared, thereby calculating the trajectory distance value between the single portrait trajectory sequence and the single reference trajectory sequence. The specific calculation is divided into the following three cases.

[0179] It should be noted that in the following three cases, a single image of a person or vehicle seen through a car window is used as an element in a single image trajectory sequence or a reference trajectory sequence. The sequence length of a single image trajectory sequence or a reference trajectory sequence is used to characterize the total number of elements it contains.

[0180] Case 1: The sequence length of a single reference trajectory sequence is greater than the sequence length of a single human portrait trajectory sequence.

[0181] In some implementations, for each of the m (m is a positive integer) elements in a single portrait trajectory sequence, the spatial position of each element is calculated, and then multiplied sequentially by the element-wise multiplication of each element with the element-wise multiplication of the n (n is a positive integer, n > m) elements in a single reference trajectory sequence to obtain a trajectory distance value, thus obtaining n-m+1 trajectory distance values. Then, the smallest trajectory distance value is selected from these n-m+1 trajectory distance values ​​as the trajectory distance value between the single portrait trajectory sequence and the single reference trajectory sequence.

[0182] Based on the explanation of calculating a single trajectory distance value, for example, the calculation of a single trajectory distance value among n-m+1 trajectory distance values ​​is performed as follows. First, a first row matrix, in the form of A1×m, is determined by horizontally arranging the spatial positions of m elements in a single portrait trajectory sequence; and a first column matrix, in the form of Bn×1, is determined by vertically arranging the spatial positions of n elements in a single reference trajectory sequence. Then, a second column matrix is ​​determined by sequentially selecting m elements from the first column matrix. Here, n-m+1 second column matrices can be determined. For a single second column matrix, the product between the first row matrix and the single second column matrix is ​​calculated, and the product is used as the single trajectory distance value.

[0183] The method described above for determining the second row matrix uses the concept of sliding window calculation.

[0184] For example, a single reference trajectory sequence can be represented as: Y=[A, B, C, D, E], where A, B, C, D, and E are the spatial coordinates (e.g., latitude and longitude coordinates) corresponding to an element (a vehicle image) in the single reference trajectory sequence. A single human portrait trajectory sequence can be represented as: X=[a, b, c], where a, b, and c are the spatial coordinates (e.g., latitude and longitude coordinates) corresponding to an element (a face image in a car window) in the single human portrait trajectory sequence. That is, aA can be used to represent the straight-line distance between two spatial coordinate points a and A. Based on the single human portrait trajectory sequence X, a first row matrix is ​​formed by arranging the data horizontally, and based on the single reference trajectory sequence Y, a first column matrix is ​​formed by arranging the data vertically. By selecting three elements from the first column matrix, (5-2+1) second column matrices are obtained, as shown in Table 3 below.

[0185]

[0186] Table 3

[0187] As shown in Table 3, the products between the first row matrix and the three second column matrices are calculated as the trajectory distance values: the first trajectory distance value distinct1 = aA + bB + cC, the second trajectory distance value distinct2 = aB + bC + cD, and the third trajectory distance value distinct3 = aC + bD + cE. Based on this, these three trajectory distance values ​​distinct1, distinct2, and distinct3 can be obtained. Then, the smallest trajectory distance value is selected as the trajectory distance value between a single reference trajectory sequence and the human figure trajectory sequence.

[0188] To more clearly illustrate the sliding window calculation concept used in the above method, see the following example, which shows the distance sequence Z between sequences X and Y.

[0189] [[aA, aB, aC, aD, aE],

[0190] Z = [bA, bB, bC, bD, bE],

[0191] [cA, cB, cC, cD, cE]]

[0192] Each value in the distance matrix Z represents the straight-line distance between any two elements corresponding to the spatial coordinates of a single human portrait trajectory sequence and a single reference trajectory sequence.

[0193] For example, if the sliding window size is 1, then the cumulative distances of each sliding window for sequence X and sequence Y are: distinct1=aA+bB+cC; distinct2=aB+bC+cD; distinct3=aC+bD+cE, that is, three trajectory distance values ​​distinct1, distinct2, and distinct3 are obtained. Then, the smallest trajectory distance value is selected as the trajectory distance value between a single reference trajectory sequence and the human image trajectory sequence.

[0194] Scenario 2: The sequence length of a single reference trajectory sequence is less than the sequence length of a single human portrait trajectory sequence.

[0195] In some implementations, the beginning or end of a single reference trajectory sequence is padded based on the first or last element of the n (n is a positive integer) elements in that sequence until the length of the single reference trajectory sequence is equal to or greater than the length of the single human image trajectory sequence. Further, if the length of the single reference trajectory sequence is equal to the length of the single human image trajectory sequence, operation three is performed; if the length of the single reference trajectory sequence is greater than the length of the single human image trajectory sequence, operation one is performed. This process yields the trajectory distance between the single human image trajectory sequence and the single reference trajectory sequence.

[0196] Scenario 3: The sequence length of a single reference trajectory sequence is equal to the sequence length of a single human portrait trajectory sequence.

[0197] In some implementations, the m elements (where m is a positive integer) of a single human image trajectory sequence are multiplied sequentially by the m elements of a single reference trajectory sequence, in the order of element arrangement, to obtain m products. Then, the sum of these m products is calculated, and the sum is used as the trajectory distance between the single human image trajectory sequence and the single reference trajectory sequence.

[0198] As one possible implementation, if there is no reference trajectory sequence in at least one (or all) vehicle trajectory sequences that matches the collection period of a single human image trajectory sequence, then it is determined that the single human image trajectory sequence does not have a trajectory distance value, or a specified value is used as the trajectory distance value corresponding to the single human image trajectory sequence.

[0199] The specified value can be, for example, positive infinity, or other values ​​set according to the actual application. Generally speaking, the specified value is a relatively large value.

[0200] It should be noted that, in the embodiments of this application, the trajectory distance value is also used to characterize the similarity between a single reference trajectory sequence and a single human image trajectory sequence. That is, the smaller the trajectory distance value, the greater the similarity, and vice versa.

[0201] Step 23: Select multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determine the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values;

[0202] In this embodiment, several trajectory distance values ​​are arranged in ascending order to obtain a sorting order of the trajectory distance values. Then, based on the sorting order, multiple trajectory distance values ​​are selected from the several trajectory distance values ​​in a selection method from front to back. Of course, a certain proportion of trajectory distance values ​​can also be selected. This solution does not specifically limit the number or method of selecting trajectory distance values.

[0203] For each selected trajectory distance value, its corresponding vehicle identifier, such as license plate number, is determined based on its respective reference trajectory sequence. Here, the license plate number can be obtained by recognizing the vehicle images that constitute the reference trajectory sequence. Of course, for multiple selected trajectory distance values, their corresponding vehicle identifiers can be the same or different.

[0204] In some implementations, a relationship is established between the first portrait file and the identified vehicle identifiers. For example, if two different vehicle identifiers are identified, a relationship is established between the first portrait file and each of the two vehicle identifiers. Of course, duplicate vehicle identifiers can also be deduplicated.

[0205] Based on the same idea, the association between multiple portrait files (excluding the first portrait file) and vehicle identifiers is obtained, thereby determining the portrait files associated with each vehicle identifier. If a vehicle identifier is associated with multiple portrait files, the file similarity value between the first portrait file and the multiple portrait files is calculated pairwise (see step 24 below for the calculation method), resulting in multiple file similarity values. Then, from the multiple file similarity values, file similarity values ​​greater than a preset threshold are selected as target similarity values, and the portrait files corresponding to the target similarity values ​​are merged (see step 25 below for the merging method).

[0206] For example, see Figure 4 As shown, the license plate numbers associated with three portrait files A, B, and C are obtained, and their association relationships are transformed to obtain the portrait files associated with license plates X and Y. Then, for portrait files associated with the same license plate, they are paired, and their respective file similarity values ​​are calculated. Files with similarity values ​​greater than a preset threshold are selected as the target similarity value, and their corresponding portrait files are merged.

[0207] Step 24: If the vehicle identification is associated with a second facial image file, then calculate the file similarity value between the first facial image file and the second facial image file;

[0208] To address the issue of multiple files for one person, the relationship between facial image files and vehicle identification is first determined based on the trajectory distance between the facial image trajectory sequence and the vehicle trajectory sequence. Therefore, under this prior condition, file similarity calculations can be performed on facial image files associated with the same vehicle identification, thereby improving the accuracy of file merging operations and effectively optimizing the phenomenon of multiple files for one person.

[0209] In this embodiment of the application, if the first portrait file is associated with a vehicle identifier, and the vehicle identifier is also associated with a second portrait file, then the file similarity value between the first portrait file and the second portrait file is calculated.

[0210] To calculate the file similarity value, firstly, based on the image feature values ​​of each element in the first image file, a first file feature value is calculated for the first image file. Then, based on the image feature values ​​of each element in the second image file, a second file feature value is calculated for the second image file. Finally, the similarity value between the first and second file feature values ​​is calculated and used as the file similarity value between the first and second image files.

[0211] For example, the first archive feature value is the centroid of the mean of the first portrait archive. The second archive feature value is the centroid of the mean of the second portrait archive. The centroid of the archive mean is the sum of the image feature values ​​of each image in the archive divided by the number of images in the archive, i.e., the average value of the image feature values. The archive similarity value is the cosine of the centroids of the mean of the two portrait archives, as shown in the following formula (1).

[0212] (1)

[0213] Here, A and B are the mean centroids of any two portrait archives. Here, A can be represented as the first archive feature value, and B can be represented as the second archive feature value.

[0214] It should be noted that the mean centroid of the above files is the X-dimensional matrix after the vehicle and card are parsed, and the cosine value of the two matrices can be used to measure the similarity between the two files.

[0215] Step 25: In response to the file similarity value being greater than a preset threshold, merge the first portrait file and the second portrait file.

[0216] The file similarity value between two portrait files is calculated based on the mean centroid of the files. A preset similarity threshold is set. If the file similarity value between two portrait files is greater than the preset similarity threshold, then the two portrait files can be considered to be merged.

[0217] For example, in this embodiment of the application, if the file similarity value between the first portrait file and the second portrait file is greater than a preset threshold, the two portrait files are merged.

[0218] For example, the above-mentioned file merging can be a way of constructing a mapping relationship and performing file merging based on the mapping relationship. In this application embodiment, it is mainly divided into the following three mapping methods.

[0219] As one possible mapping method, a first portrait file is mapped to a second portrait file to obtain a first mapping relationship, and the first mapping relationship is saved in the second portrait file. For example, in the embodiments of this application, the first portrait file A and the second portrait file B can be combined, so A can be mapped to B to form the same file, and the first mapping relationship (A—>B) is saved in B for traceability.

[0220] As one possible mapping method, the second portrait file is mapped to the first portrait file to obtain a second mapping relationship, and the second mapping relationship is saved in the first portrait file. For example, in the embodiments of this application, the first portrait file A and the second portrait file B can be combined, then B can be mapped to A to form the same file, and at the same time the second mapping relationship (B—>A) is saved in A for traceability.

[0221] As one possible mapping method, the first portrait file is merged with the second portrait file, and the second portrait file is merged with the third portrait file. Then, the first portrait file is mapped to the second portrait file to obtain a first mapping relationship, and the third portrait file is mapped to the second portrait file to obtain a third mapping relationship. Both the first and third mapping relationships are saved in the second portrait file. For example, in this embodiment, the first portrait file A and the second portrait file B can be merged, so A can be mapped to B to form the same file, and the first mapping relationship (A->B) is saved in B for traceability. Furthermore, the third portrait file C and the second portrait file B can be merged, so C can be mapped to B to form the same file, and the third mapping relationship (C->B) is saved in B for traceability. In this way, the first portrait file A, the second portrait file B, and the third portrait file C can be mutually mapped, meaning they can be merged into one file.

[0222] On the one hand, this solution utilizes the similarity between facial image trajectories and vehicle trajectories to obtain target files from massive facial image archives, thereby reducing the file similarity threshold and optimizing the issue of multiple facial image archives.

[0223] On the other hand, this solution provides a method for calculating the distance between trajectory sequences of unequal lengths. Specifically, it calculates the distance between two trajectory sequences by sequence completion and sliding window calculation, and then judges the similarity between the two trajectory sequences.

[0224] Based on the same inventive concept, this application also provides a device for merging multiple service levels, used to merge multiple service levels for one person, reducing the rate of multiple service levels. See [link to related document]. Figure 5 The device includes:

[0225] Module 51 acquires M sequences of human image trajectories generated based on the first human image profile; where M is an integer greater than 0.

[0226] The first calculation module 52 calculates the trajectory distance value between each of the M human image trajectory sequences and the reference trajectory sequence to obtain several trajectory distance values; wherein, the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period;

[0227] The determining module 53 selects multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determines the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values;

[0228] The second calculation module 54 calculates the file similarity value between the first image file and the second image file if the vehicle identifier is associated with a second image file.

[0229] The merging module 55, in response to the file similarity value being greater than a preset threshold, merges the first portrait file and the second portrait file.

[0230] In one possible implementation, the acquisition module 51, which acquires M sequences of facial trajectories generated based on the first facial profile, is specifically used for:

[0231] Obtain a first portrait file of the first target object; wherein the first portrait file includes at least: multiple facial images of the first target object captured by car windows;

[0232] According to the image acquisition time, multiple face images of car windows in the first portrait file are sorted to obtain the first portrait trajectory sequence;

[0233] In the first trajectory sequence, duplicate images of faces in the car window that are continuously in the same spatial position are deduplicated to obtain the second portrait trajectory sequence.

[0234] Based on the preset M segmentation time periods, the face images in the car window of the second portrait trajectory sequence are divided to obtain M portrait trajectory sequences.

[0235] In one possible implementation, the first calculation module 52, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0236] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0237] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0238] In response to the fact that the sequence length of the reference trajectory sequence is greater than the sequence length of the single human portrait trajectory sequence, for each of the m elements in the single human portrait trajectory sequence, the trajectory distance value is calculated by sequentially multiplying the spatial position of each element with the spatial position of each of the n elements in the reference trajectory sequence, resulting in n-m+1 trajectory distance values; where m and n are integers greater than 0, and m is less than n;

[0239] From the n-m+1 trajectory distance values, the smallest trajectory distance value is selected as the trajectory distance value between the single human portrait trajectory sequence and the reference trajectory sequence.

[0240] In one possible implementation, for each of the m elements in the single human portrait trajectory sequence, the trajectory distance values ​​are obtained by calculating the sum of the spatial positions of each element and the spatial positions of individual elements in the n elements of the reference trajectory sequence, in order, resulting in n-m+1 trajectory distance values. The first calculation module 52 is further configured to:

[0241] For a single trajectory distance value among the n-m+1 trajectory distance values, the following method is used for calculation:

[0242] Determine the first row matrix composed of the horizontal arrangement of the spatial positions of m elements in the single human portrait trajectory sequence;

[0243] Determine the second column matrix by selecting m elements sequentially from the first column matrix; wherein the number of the second column matrices is n-m+1.

[0244] The product of the first row matrix and the second column matrix is ​​calculated as the distance value of the single trajectory.

[0245] In one possible implementation, the first calculation module 52, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0246] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0247] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0248] In response to the fact that the sequence length of the reference trajectory sequence is less than the sequence length of the single human portrait trajectory sequence, the reference trajectory sequence is padded based on the first or last element of the n elements in the reference trajectory sequence; where n is an integer greater than 0.

[0249] The trajectory distance between each portrait sequence and the reference trajectory sequence is calculated until the sequence length of the reference trajectory sequence is equal to or greater than the sequence length of the single portrait trajectory sequence.

[0250] In one possible implementation, the first calculation module 52, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0251] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0252] From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence;

[0253] In response to the fact that the sequence length of the reference trajectory sequence is equal to the sequence length of the single human portrait trajectory sequence, the m elements of the single human portrait trajectory sequence are multiplied by the m elements of the reference trajectory sequence in the order of element arrangement to obtain m products;

[0254] Calculate the sum of the m products, and use the sum as the trajectory distance between the single human image trajectory sequence and the reference trajectory sequence.

[0255] In one possible implementation, the first calculation module 52, in calculating the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence, is specifically used for:

[0256] For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations:

[0257] If at least one vehicle trajectory sequence does not have a reference trajectory sequence that matches the acquisition time period of the single human image trajectory sequence, then it is determined that the single human image trajectory sequence does not have a trajectory distance value, or a specified value is used as the trajectory distance value corresponding to the single human image trajectory sequence; wherein, the specified value includes at least positive infinity.

[0258] In one possible implementation, the determination module 53, which selects multiple trajectory distance values ​​that satisfy preset conditions from the plurality of trajectory distance values, is specifically used for:

[0259] Arrange the trajectory distance values ​​in ascending order to obtain the sorting order of the several trajectory distance values;

[0260] Based on the arrangement order, multiple trajectory distance values ​​are selected from the plurality of trajectory distance values ​​in a selection manner from front to back.

[0261] In one possible implementation, after determining the vehicle identifiers corresponding to each of the plurality of trajectory distance values, the determining module 53 is further configured to:

[0262] Associate the first portrait file with the vehicle identification.

[0263] In one possible implementation, before calculating the file similarity value between the first and second image files if the vehicle identifier is associated with a second image file, the second calculation module 54 is further configured to:

[0264] If the vehicle identifier is associated with multiple portrait files, then the file similarity value between the first portrait file and the multiple portrait files is calculated in pairs to obtain multiple file similarity values;

[0265] From the multiple file similarity values, file similarity values ​​greater than a preset threshold are selected as target similarity values, and the portrait files corresponding to the target similarity values ​​are merged.

[0266] In one possible implementation, the second calculation module 54, in calculating the file similarity value between the first portrait file and the second portrait file, is specifically used for:

[0267] Based on the image feature values ​​of each element in the first portrait file, calculate the first file feature value of the first portrait file;

[0268] Based on the image feature values ​​of each element in the second portrait file, calculate the second file feature value of the second portrait file;

[0269] Calculate the similarity value between the first archive feature value and the second archive feature value, and use it as the archive similarity value between the first portrait archive and the second portrait archive.

[0270] In one possible implementation, the merging module 55, which merges the first portrait file and the second portrait file, is specifically used for:

[0271] Map the first portrait file to the second portrait file to obtain a first mapping relationship, and save the first mapping relationship to the second portrait file; or

[0272] The second portrait file is mapped onto the first portrait file to obtain a second mapping relationship, and the second mapping relationship is saved into the first portrait file.

[0273] Based on the above device, by introducing a reference trajectory sequence, namely a vehicle trajectory sequence that matches the collection time period with the corresponding human image trajectory sequence, multi-file merging based on human image trajectory sequence and reference trajectory sequence can be achieved to solve the problem of optimizing multiple human image files and improve the accuracy of multi-file merging.

[0274] Based on the same inventive concept, this application also provides an electronic device that can realize the function of the aforementioned multi-level merging device. (Refer to...) Figure 6 The electronic device includes:

[0275] At least one processor 61 and a memory 62 connected to at least one processor 61. In this embodiment, the specific connection medium between the processor 61 and the memory 62 is not limited. Figure 6 The example shown is the connection between processor 61 and memory 62 via bus 60. Bus 60 is... Figure 6 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Bus 60 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 6 The term 61 is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, the processor 61 can also be called a controller; there is no restriction on the name.

[0276] In this embodiment, memory 62 stores instructions executable by at least one processor 61. By executing the instructions stored in memory 62, at least one processor 61 can perform the multi-file merging method described above. Processor 61 can implement... Figure 5 The functions of each module in the device / system shown.

[0277] The processor 61 is the control center of the device / system. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 62 and calling data stored in memory 62, it can monitor the various functions and data processing of the device / system as a whole.

[0278] In one possible design, processor 61 may include one or more processing units. Processor 61 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 61. In some embodiments, processor 61 and memory 62 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0279] Processor 61 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the multi-level merging method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0280] Memory 62, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 62 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 62 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 62 can also be a circuit or any other device / system capable of implementing storage functions for storing program instructions and / or data.

[0281] By designing and programming the processor 61, the code corresponding to the multi-level merging method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute it during operation. Figure 2 The steps of the multi-file merging method in the illustrated embodiment are described below. How to design and program the processor 61 is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0282] Based on the same inventive concept, embodiments of this application also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the multi-file merging method described above.

[0283] In some possible implementations, various aspects of the multi-file merging method provided in this application can also be implemented in the form of a program product, which includes program code that, when the program product is run on a device, causes the control device to perform the steps in the multi-file merging method according to the various exemplary embodiments of this application described above.

[0284] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus / systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0285] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0286] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0287] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0288] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for merging multiple files, characterized in that, The method includes: Obtain a first portrait file of the first target object; wherein the first portrait file includes at least: multiple facial images of the first target object captured by car windows; According to the image acquisition time, multiple face images of car windows in the first portrait file are sorted to obtain the first portrait trajectory sequence; In the first portrait trajectory sequence, duplicate images of faces in the car window that are continuously in the same spatial position are deduplicated to obtain the second portrait trajectory sequence. Based on the preset M segmentation time periods, the face images in the car window of the second portrait trajectory sequence are divided to obtain M portrait trajectory sequences; where M is an integer greater than 0; For the M human image trajectory sequences, the trajectory distance value between each human image trajectory sequence and the reference trajectory sequence is calculated to obtain several trajectory distance values; wherein, the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period; Select multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determine the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values; If the vehicle identification is associated with a second facial image file, then calculate the file similarity value between the first facial image file and the second facial image file; In response to the file similarity value being greater than a preset threshold, the first portrait file and the second portrait file are merged.

2. The method as described in claim 1, characterized in that, The calculation of the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes: For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations: From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence; In response to the fact that the sequence length of the reference trajectory sequence is greater than the sequence length of the single human portrait trajectory sequence, for each of the m elements in the single human portrait trajectory sequence, the trajectory distance value is calculated by sequentially multiplying the spatial position of each element with the spatial position of each of the n elements in the reference trajectory sequence, resulting in n-m+1 trajectory distance values; where m and n are integers greater than 0, and m is less than n; From the n-m+1 trajectory distance values, the smallest trajectory distance value is selected as the trajectory distance value between the single human portrait trajectory sequence and the reference trajectory sequence.

3. The method as described in claim 2, characterized in that, For each of the m elements in the single human image trajectory sequence, the trajectory distance value is obtained by calculating the sum of the spatial positions of each element and the spatial positions of individual elements in the n elements of the reference trajectory sequence, resulting in n-m+1 trajectory distance values, including: For a single trajectory distance value among the n-m+1 trajectory distance values, the following method is used for calculation: Determine the first row matrix composed of the horizontal arrangement of the spatial positions of m elements in the single human portrait trajectory sequence; Determine the first column matrix, which is composed of the vertical arrangement of the spatial positions of n elements in a single reference trajectory sequence; Determine the second column matrix by selecting m elements sequentially from the first column matrix; wherein the number of the second column matrices is n-m+1. The product of the first row matrix and the second column matrix is ​​calculated as the distance value of the single trajectory.

4. The method as described in claim 1, characterized in that, The calculation of the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes: For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations: From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence; In response to the fact that the sequence length of the reference trajectory sequence is less than the sequence length of the single human portrait trajectory sequence, the reference trajectory sequence is padded based on the first or last element of the n elements in the reference trajectory sequence; where n is an integer greater than 0. The trajectory distance between each portrait sequence and the reference trajectory sequence is calculated until the sequence length of the reference trajectory sequence is equal to or greater than the sequence length of the single portrait trajectory sequence.

5. The method as described in claim 1, characterized in that, The calculation of the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes: For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations: From at least one vehicle trajectory sequence, determine a reference trajectory sequence that matches the collection period with the single human image trajectory sequence; In response to the fact that the sequence length of the reference trajectory sequence is equal to the sequence length of the single human portrait trajectory sequence, the m elements of the single human portrait trajectory sequence are multiplied by the m elements of the reference trajectory sequence in the order of element arrangement to obtain m products; Calculate the sum of the m products, and use the sum as the trajectory distance between the single human image trajectory sequence and the reference trajectory sequence.

6. The method as described in claim 1, characterized in that, The calculation of the trajectory distance value between each portrait trajectory sequence and the reference trajectory sequence includes: For a single portrait trajectory sequence among the M portrait trajectory sequences, perform the following operations: If at least one vehicle trajectory sequence does not have a reference trajectory sequence that matches the acquisition time period of the single human image trajectory sequence, then it is determined that the single human image trajectory sequence does not have a trajectory distance value, or a specified value is used as the trajectory distance value corresponding to the single human image trajectory sequence; wherein, the specified value includes at least positive infinity.

7. The method as described in claim 1, characterized in that, The step of selecting multiple trajectory distance values ​​that satisfy preset conditions from the plurality of trajectory distance values ​​includes: Arrange the trajectory distance values ​​in ascending order to obtain the sorting order of the several trajectory distance values; Based on the arrangement order, multiple trajectory distance values ​​are selected from the plurality of trajectory distance values ​​in a selection manner from front to back.

8. The method as described in claim 1, characterized in that, After determining the vehicle identifiers corresponding to each of the plurality of trajectory distance values, the method further includes: Associate the first portrait file with the vehicle identification.

9. The method as described in claim 1 or 8, characterized in that, Before calculating the file similarity value between the first and second image files if the vehicle identifier is associated with a second image file, the method further includes: If the vehicle identifier is associated with multiple portrait files, then the file similarity value between the first portrait file and the multiple portrait files is calculated in pairs to obtain multiple file similarity values; From the multiple file similarity values, file similarity values ​​greater than a preset threshold are selected as target similarity values, and the portrait files corresponding to the target similarity values ​​are merged.

10. The method as described in claim 1, characterized in that, The calculation of the file similarity value between the first portrait file and the second portrait file includes: Based on the image feature values ​​of each element in the first portrait file, calculate the first file feature value of the first portrait file; Based on the image feature values ​​of each element in the second portrait file, calculate the second file feature value of the second portrait file; Calculate the similarity value between the first archive feature value and the second archive feature value, and use it as the archive similarity value between the first portrait archive and the second portrait archive.

11. The method as described in claim 1, characterized in that, The merging of the first portrait file and the second portrait file includes: Map the first portrait file to the second portrait file to obtain a first mapping relationship, and save the first mapping relationship to the second portrait file; or The second portrait file is mapped onto the first portrait file to obtain a second mapping relationship, and the second mapping relationship is saved into the first portrait file.

12. A device for merging multiple gears, characterized in that, The device includes: The acquisition module acquires a first portrait file of the first target object; wherein, the first portrait file includes at least: multiple facial images of the first target object captured by car windows; According to the image acquisition time, multiple face images of car windows in the first portrait file are sorted to obtain the first portrait trajectory sequence; In the first portrait trajectory sequence, duplicate images of faces in the car window that are continuously in the same spatial position are deduplicated to obtain the second portrait trajectory sequence. Based on the preset M segmentation time periods, the face images in the car window of the second portrait trajectory sequence are divided to obtain M portrait trajectory sequences; where M is an integer greater than 0; The first calculation module calculates the trajectory distance value between each of the M human image trajectory sequences and the reference trajectory sequence to obtain several trajectory distance values; wherein, the reference trajectory sequence is the vehicle trajectory sequence that matches the corresponding human image trajectory sequence during the collection period; The determination module selects multiple trajectory distance values ​​that meet preset conditions from the plurality of trajectory distance values, and determines the vehicle identifier of the reference trajectory sequence corresponding to each of the multiple trajectory distance values; The second calculation module calculates the file similarity value between the first image file and the second image file if the vehicle identifier is associated with a second image file. The merging module merges the first portrait file and the second portrait file in response to the file similarity value being greater than a preset threshold.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method steps of any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-11.

Images