Target retrieval method and system, terminal device, cloud server, medium, device

Abstract

The disclosed embodiments disclose a target retrieval method and system, a terminal device, a cloud server, a medium, and a device, wherein the method comprises: processing a collected target image through a first model to obtain structured data; determining model-related information corresponding to the first model; generating encrypted data based on the structured data and the model-related information; and sending the encrypted data to a cloud server. In this embodiment, since structured data rather than the original snapshot or video stream is reported to the cloud server, bandwidth occupancy is reduced, cloud server resource usage is reduced, costs are saved, and retrieval efficiency is improved. Furthermore, since the structured data does not include specific target information, the security and privacy of the target data are guaranteed.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and in particular to a target retrieval method and system, terminal equipment, cloud server, medium, and equipment. Background Art

[0002] Target retrieval refers to the process of identifying a specific target (or a type of target) from multiple targets (or multiple types of targets) and its corresponding target. At present, the main target retrieval method is to capture the target image with the front-end acquisition device and upload it to the server for identification and retrieval. With the large-scale use of target retrieval and the increasing popularity of front-end intelligent devices, more and more images are uploaded to the server or cloud, which puts great demands on bandwidth and server resources. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure is proposed. The embodiments of the present disclosure provide a target retrieval method and system, a terminal device, a cloud server, a medium, and a device.

[0004] According to one aspect of an embodiment of the present disclosure, a target retrieval method is provided, which is applied to a terminal device and includes:

[0005] Processing the acquired target image through the first model to obtain structured data;

[0006] Determine model-related information corresponding to the first model;

[0007] Generate encrypted data based on the structured data and the model-related information;

[0008] The encrypted data is sent to a cloud server.

[0009] According to another aspect of an embodiment of the present disclosure, a target retrieval method is provided, which is applied to a cloud server and includes:

[0010] Receiving encrypted data sent by a terminal device;

[0011] Based on the encrypted data, determine structured data and model-related information; wherein the structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model;

[0012] Determine a feature database corresponding to the structured data based on the model related information;

[0013] The record data corresponding to the target is retrieved from the feature database based on the structured data.

[0014] According to another aspect of an embodiment of the present disclosure, a terminal device is provided, including:

[0015] An image processing module, used to process the collected target image through the first model to obtain structured data;

[0016] An information determination module, used to determine model-related information corresponding to the first model provided by the image processing module;

[0017] a data generating module, configured to generate encrypted data based on the structured data obtained by the image processing module and the model-related information determined by the information determining module;

[0018] The data sending module is used to send the encrypted data obtained by the data generating module to the cloud server.

[0019] According to another aspect of the embodiments of the present disclosure, a cloud server is provided, including:

[0020] A data receiving module, used for receiving encrypted data sent by a terminal device;

[0021] A data parsing module, configured to determine structured data and model-related information based on the encrypted data received by the data receiving module; wherein the structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model;

[0022] A database search module, used to determine a feature database corresponding to the structured data based on the model related information determined by the data analysis module;

[0023] The data matching module is used to retrieve the record data corresponding to the target from the feature database determined by the database search module based on the structured data determined by the data parsing module.

[0024] According to another aspect of the embodiments of the present disclosure, there is provided a target retrieval system, comprising: a cloud server as described in the above embodiments and at least one terminal device as described in the above embodiments.

[0025] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the target retrieval method described in the above embodiment.

[0026] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0027] processor;

[0028] a memory for storing instructions executable by the processor;

[0029] The processor is used to read the executable instructions from the memory and execute the instructions to implement the target retrieval method described in the above embodiment.

[0030] Based on a target retrieval method and system, terminal device, cloud server, medium, and device provided by the above-mentioned embodiments of the present disclosure, the collected target image is processed through a first model to obtain structured data; model-related information corresponding to the first model is determined; encrypted data is generated based on the structured data and the model-related information; and the encrypted data is sent to the cloud server. Since structured data rather than the original snapshot or video stream is reported to the cloud server in this embodiment, bandwidth occupancy is reduced; the use of cloud server resources is reduced, costs are saved, and retrieval efficiency is improved; and since the structured data does not include specific target information, the security and privacy of the target data are guaranteed.

[0031] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0033] Figure 1 It is a schematic diagram of the framework of a target retrieval system provided by an exemplary embodiment of the present disclosure.

[0034] Figure 2 Another exemplary embodiment of the present disclosure provides a flowchart of a target retrieval system implementing target retrieval.

[0035] Figure 3 It is a flowchart of a target retrieval method provided by an exemplary embodiment of the present disclosure.

[0036] Figure 4 It is a flowchart of a target retrieval method provided by another exemplary embodiment of the present disclosure.

[0037] Figure 5 This disclosure Figure 3 A schematic flow chart of step 301 in the embodiment shown.

[0038] Figure 6 It is a flowchart of a target retrieval method provided by another exemplary embodiment of the present disclosure.

[0039] Figure 7This disclosure Figure 6 A flowchart of step 603 in the embodiment shown is shown.

[0040] Figure 8 This disclosure Figure 7 A flowchart of step 6033 in the embodiment shown.

[0041] Fig. 9 It is a schematic diagram of the structure of a terminal device provided by an exemplary embodiment of the present disclosure.

[0042] Fig.10 It is a structural diagram of a terminal device provided by another exemplary embodiment of the present disclosure.

[0043] Fig.11 It is a schematic diagram of the structure of a cloud server provided by an exemplary embodiment of the present disclosure.

[0044] Fig.12 It is a schematic diagram of the structure of a cloud server provided by another exemplary embodiment of the present disclosure.

[0045] Fig.13 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0047] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0048] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0049] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0050] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0051] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0052] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0053] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0054] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0055] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0056] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0057] The disclosed embodiments can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0058] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0059] Application Overview

[0060] In the process of implementing the present disclosure, the inventors found that in the prior art, the processing performance of the cloud is enhanced by horizontal expansion, increasing server resources, and cluster deployment to support the access of more terminal devices, but this technical solution has at least the following problems: unlimited expansion will lead to increased costs and low cost performance.

[0061] Exemplary Systems

[0062] The target retrieval system provided by the present disclosure includes a terminal device for performing data collection and calculation and a cloud server for retrieval, wherein the data collection and calculation of the terminal device is realized by an intelligent device with AI capabilities, such as an intelligent network camera, a panel machine, and a vehicle-mounted device, etc., to detect, track, and capture targets in the target area, and select target images with better quality (for example, when the target is a human face, the face quality may include: whether it is frontal, the occlusion area, the image clarity, etc.), and then use a deep learning algorithm to extract target attributes, target key points, and target feature information from the captured image. Optionally, when the target is a human face, the face attributes include information such as gender, age, and skin color (attributes such as gender and age can be used for retrieval, and can be classified, stored, and displayed according to the attributes during the storage process), the face key points include the location information of key areas of the face, such as the nose, eyebrows, mouth, eyes, etc. (the role of identifying face key points is to save information), and the face feature vector is a representation of the high-dimensional space of the face, which can be 256 dimensions or 128 dimensions, etc., and is used for face comparison and retrieval. Afterwards, the target attributes, target key points, and target feature information are encapsulated in a structure recognizable by the cloud server (format encapsulation so that the cloud server can recognize it), encapsulated into a structured message and compressed and encrypted, where the structured message also contains the version information of the recognition model (which can be a neural network) used by the device, and this model version can be associated with the feature library of the cloud server; different types of terminal devices are connected to the cloud server through the cloud data access layer, and the terminal devices can report the generated structured data to the cloud server through the access platform for retrieval; cloud retrieval decompresses and decrypts the structured data, extracts the target features, model version, and recognition type (the type set in the terminal device) in the structured data, selects the corresponding retrieval service according to the recognition type, selects the corresponding feature library according to the model version, and uses the target feature information to search and compare in the feature library, and outputs the recognition results.

[0063] In the disclosed embodiments, since the terminal device reports structured data rather than the original snapshot or video stream, the bandwidth utilization is reduced. Since the structured data does not contain specific facial information, the security and privacy of the facial data are guaranteed. The disclosed embodiments place part of the calculation on the terminal device, reducing the cost of the cloud server. The terminal device and the cloud server are interacted through the model version, and the cloud server can communicate with terminal devices with different recognition models. Different types of terminal devices can retrieve the same feature library in the cloud server as long as they use the same recognition model. The target retrieval system provided by the disclosed embodiments can be applied to the recognition of various objects, such as faces, bodies, commodities, and license plates. As long as the terminal device integrates the corresponding recognition model and the cloud server has a retrieval library of the corresponding model, the terminal device can be connected to the cloud server for recognition and retrieval.

[0064] Figure 1 FIG. 1 is a schematic diagram of a target retrieval system provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the target retrieval system can be divided into a device side (including at least one terminal device 11) and a cloud side (including a cloud server 12); wherein, in the process of realizing target retrieval by the target retrieval system, the functions of the terminal device 11 and the cloud server 12 include the following:

[0065] The terminal device 11 has detection, tracking and capture recognition capabilities through an embedded AI chip and an integrated target (e.g., face, etc.) recognition model. The terminal device may include but is not limited to the following types: smart capture cameras, smart panel machines, and smart vehicle-mounted devices.

[0066] The terminal device captures the targets in the target area, extracts the target attributes, key points and features, generates structured data and uploads it to the cloud.

[0067] The cloud server in the cloud parses the structured data reported by the terminal device and selects the corresponding feature library according to the model version used by the terminal device.

[0068] The cloud server stores feature libraries of different model versions. When registering a feature library, you need to specify the model version used by the feature library. When retrieving the feature library, you need to specify the model version.

[0069] The target feature information reported by the terminal device is compared with the target record in the corresponding feature library to find the most similar target record.

[0070] Figure 2 Another exemplary embodiment of the present disclosure provides a flow chart of a target retrieval system for implementing target retrieval. The target retrieval system mainly includes two parts: the device end and the cloud end. Figure 2As shown in Figure 1, the target retrieval process includes the following steps:

[0071] 201: The terminal device detects and tracks the target in the target area through the built-in AI chip to obtain the tracking trajectory of the target.

[0072] Among them, the AI ​​chip can execute a deep learning algorithm to detect targets appearing in the target area (for example, face detection) to determine the location of the target. Based on the target position obtained through multiple detections, the tracking trajectory of the target can be obtained to achieve tracking of the target.

[0073] 202: The terminal device captures the detected and tracked target based on the tracking trajectory of the target, and selects one or a group of snapshots with better quality; wherein the selection may include but is not limited to: selecting an image with better quality from multiple images, and the quality includes but is not limited to: image clarity, the size of the target in the image, etc. For example, when the target is a human face, the image quality may also include: whether it is a frontal face and other information.

[0074] 203: The terminal device extracts target attributes, key points and features from the captured image based on the deep learning algorithm integrated on the terminal device.

[0075] 204: When reporting data, the terminal device reports the type of the recognition model currently integrated in the terminal device to the cloud server. The cloud server selects different services to process the data reported by the terminal device according to the recognition type.

[0076] Among them, the terminal device (each terminal device can integrate multiple recognition models) realizes the recognition of different targets by integrating different types of recognition models. For example, the integrated face recognition model can realize the recognition of faces, the integrated human body model can realize the recognition of human bodies, and the integrated vehicle model can realize the recognition of vehicles, etc.

[0077] 205: The model integrated in the current terminal device is marked by the model version identifier; when the terminal device reports data, the model version number information needs to be reported, and the cloud server associates the feature library according to the model version number.

[0078] Among them, different terminal devices have great differences in performance and computing power. Therefore, the models integrated into different terminal devices may be different. Terminal devices with strong performance can integrate models with relatively large computing power, and terminal devices with weak performance can integrate models with relatively small computing power. In addition, the model is also constantly iterated and updated (updated according to the update of the model in the cloud server).

[0079] 206: The structured data, model version information, identification type and other data calculated in step 203 are packaged, encrypted and compressed, and uploaded to the cloud server.

[0080] 207: The terminal device establishes a connection with the cloud server through relevant protocols and uploads the encrypted data generated on the terminal device to the cloud server; among them, the cloud data access layer is a universal data access platform. As long as it meets the relevant protocol requirements and passes the relevant authorization and authentication, the device can be connected to the cloud server. The cloud server will manage and maintain the status of the terminal device.

[0081] 208: Decompress and decrypt the data reported by the terminal device to obtain structured data, and select a corresponding retrieval service according to the identification type in the structured data; wherein, in the cloud server, each type of retrieval corresponds to a service, and the message reported by each terminal device will be balanced to different retrieval services according to the type.

[0082] 209: Input the structured data into the corresponding service (eg, face search service, body search service, product search service, etc.) according to the recognition type.

[0083] 210: When the terminal device reports data for retrieval, the face library to be retrieved is determined based on the name of the face library associated with the terminal device (each terminal device corresponds to at least one face library) and the model version number information on the terminal device.

[0084] Among them, the cloud server creates a feature library (for example Figure 2 In the example above, when face is used as the retrieval target, the feature library is the face library. When face is used as the retrieval target, the model version corresponding to the features in the feature library needs to be specified. The model version needs to be specified when searching. Different model versions correspond to different retrieval parameters and thresholds.

[0085] 211: A cloud server face library is uniquely identified by the face library name + model version number. When searching, the library to be searched needs to be found based on the face library name and model version number. Among them, the facial features extracted by different models are very different, so to identify a face library, it is necessary to specify not only the name of the face library but also the model version number corresponding to the facial features in the face library.

[0086] 212: By comparing the features to be retrieved with the features in the face database to be retrieved, the record with the shortest distance is found. It can also be determined whether the shortest distance is less than a threshold, and the record with the shortest distance is used as the retrieval result.

[0087] Exemplary Methods

[0088] Figure 3FIG. 1 is a flow chart of a target search method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to a terminal device, such as Figure 3 As shown, the following steps are included:

[0089] Step 301: Process the collected target image through a first model to obtain structured data.

[0090] The target object can be acquired by a camera device provided on the terminal device, and optionally, the target image can be processed by a deep learning algorithm to obtain structured data; for example, Figure 2 The process shown in steps 201-203 in the provided embodiment obtains structured data.

[0091] Step 302: determine model-related information corresponding to the first model.

[0092] In one embodiment, reference may be made to Figure 2 In the technical content shown in step 204 of the provided embodiment, multiple recognition models are integrated in each terminal device, and the first model can be any one of the multiple recognition models. In this embodiment, it is necessary to determine the relevant information of the first model for processing the target image, for example, the relevant information of the first model is the type information of the first model, etc. The type information of the model may include but is not limited to: face recognition model, human body recognition model, etc.

[0093] Step 303: Generate encrypted data based on the structured data and model-related information.

[0094] Optionally, the method for obtaining encrypted data in this embodiment can refer to Figure 2 As shown in step 206 in the provided embodiment, the structured data and model-related information are processed, for example, by packaging, encryption, compression, etc. to obtain encrypted data.

[0095] Step 304, sending the encrypted data to the cloud server.

[0096] In this implementation, only partial processing is performed on the captured target image in the terminal device, and the processed encrypted data is sent to the cloud server. Through the processing of the above steps, the size of data transmitted between the terminal device and the cloud server is reduced, the transmission efficiency is improved, and the processing pressure of the cloud server is reduced.

[0097] The above-mentioned embodiment of the present disclosure provides a target retrieval method, which processes the collected target image through a first model to obtain structured data; determines the model-related information corresponding to the first model; generates encrypted data based on the structured data and the model-related information; and sends the encrypted data to a cloud server. Since the present embodiment reports structured data to the cloud server instead of the original snapshot or video stream, it reduces bandwidth occupancy, reduces the use of cloud server resources, saves costs, and improves retrieval efficiency. Moreover, since the structured data does not include specific target information, the security and privacy of the target data are guaranteed.

[0098] Figure 4 FIG. 1 is a flow chart of a target search method provided by another exemplary embodiment of the present disclosure. This embodiment can be applied to a terminal device, such as Figure 4 As shown, the following steps are included:

[0099] Step 301: Process the collected target image through a first model to obtain structured data.

[0100] Step 3021, determine the feature library name, model type and model version identifier of the first model.

[0101] Among them, each terminal device corresponds to a feature library name, and each feature library name corresponds to multiple feature libraries. Therefore, if the terminal device is known, the feature library name can be determined; the model type (such as face recognition model or body recognition model, etc.) is determined by identifying the first model; and the determination of the model version identifier can refer to Figure 2 In the technical content shown in step 205 of the provided embodiment, the model is continuously iterated and updated in the terminal device, and a different model version identifier will be obtained after each update. The model version identifier is used to indicate the model integrated in the current terminal device.

[0102] Step 3022: Determine model-related information corresponding to the first model based on the feature library name, model type, and model version identifier.

[0103] Step 3031, encapsulate the structured data, feature library name, model type and model version identifier according to a set format to obtain encapsulated data in the set format.

[0104] The setting format corresponds to the recognition format of the cloud server.

[0105] The purpose of encapsulation in this embodiment is to make it recognizable to the cloud server. Therefore, the structured data, feature library name, model type and model version identifier are encapsulated into a format recognizable to the cloud server through format encapsulation to avoid the situation where the data content cannot be recognized due to format problems.

[0106] Step 3032: compress and encrypt the encapsulated data to obtain processed encrypted data.

[0107] Among them, data compression refers to a technical method of reducing the amount of data to reduce storage space, improve its transmission, storage and processing efficiency, or reorganize the data according to a certain algorithm to reduce data redundancy and storage space without losing useful information; data compression includes lossy compression and lossless compression. This embodiment can use any compression method in the prior art to compress the encapsulated data. The encryption processing proposed in this embodiment can be any encryption method in the prior art, for example, key encryption (such as symmetric encryption, asymmetric encryption, etc.), etc. The embodiment of this application does not limit the specific compression and encryption technology.

[0108] Step 304, sending the encrypted data to the cloud server.

[0109] In this embodiment, compression processing reduces the bandwidth occupied by the encapsulated data and the space occupied in the cloud server, thereby improving data transmission efficiency, and there is no need to enhance the processing performance of the cloud server, thereby reducing processing costs; encryption processing enhances data security. In addition, in this embodiment, since different terminal devices have great differences in performance and computing power, the models integrated by different terminal devices may be different. Terminal devices with strong performance may integrate models with relatively large computing power, and terminal devices with weak performance may integrate models with relatively small computing power; therefore, when reporting data, the terminal device reports the type of recognition model currently integrated by the terminal device to the cloud server, and the cloud server selects different services according to the recognition type to process the data reported by the terminal device.

[0110] like Figure 5 As shown in the above Figure 3 Based on the illustrated embodiment, step 301 may include the following steps:

[0111] Step 3011: determine a first model from at least one model included in the terminal device.

[0112] Among them, at least one model corresponds to at least one model type, and models of different model types process the target image differently.

[0113] Optionally, each model corresponds to a model type, and the model type may include but is not limited to: face recognition model, body recognition model, vehicle recognition model, etc. Correspondingly, the face recognition model can realize the recognition of faces, the body recognition model can realize the recognition of bodies, the vehicle recognition model can realize the recognition of vehicles, etc.

[0114] Step 3012: extract features of the target image using the determined first model to obtain target features; and use the target features as structured data.

[0115] In this embodiment, a deep learning algorithm integrated on a terminal device is used to extract target attributes, key points, and features from the captured image. Optionally, the structured data may include target attributes and key point information in addition to target features.

[0116] In some optional embodiments, the method provided in this embodiment further includes:

[0117] In response to detecting a model version update request, performing a version update on the corresponding second model according to the model version update request;

[0118] The updated model version identifier is saved and sent to the cloud server to update the feature database in the cloud server.

[0119] In this embodiment, the model is continuously iteratively updated. When the template version in the terminal device is updated according to the version update request, the updated model version identifier can be obtained. At this time, the feature database in the cloud server still corresponds to the model version identifier before the update. Therefore, it is necessary to send the updated model version identifier to the cloud server to obtain the feature database obtained by processing the model corresponding to the updated model version identifier. All features in the feature database are obtained based on the updated model processing.

[0120] In some optional embodiments, the method provided in this embodiment further includes:

[0121] An image or video including a target is acquired; target area detection and quality screening are performed on at least one frame of video image in the image or video to obtain a target image.

[0122] In order to realize the retrieval of the target in this embodiment, it is first necessary to obtain the target image. Optionally, the image or video including the target can be collected by the image acquisition device in the terminal device to realize the detection and tracking of the target in the target area. In order to improve the efficiency of subsequent target retrieval, the target needs to be displayed in the obtained target image with high quality. Optionally, through quality screening, one or a group of images with good quality are obtained as the target image, and the quality evaluation may include but is not limited to: clarity, proportion of the target, etc.

[0123] Figure 6 FIG. 1 is a flowchart of a target search method provided by another exemplary embodiment of the present disclosure. This embodiment can be applied on a cloud server, such as Figure 6 As shown, the following steps are included:

[0124] Step 601: receiving encrypted data sent by a terminal device.

[0125] Optionally, the process of cloud server receiving encrypted data can be found in Figure 2 As shown in step 207 of the provided embodiment, the terminal device establishes a connection with the cloud server through a relevant protocol and uploads the encrypted data generated on the terminal device to the cloud server.

[0126] Step 602: Determine structured data and model-related information based on the encrypted data.

[0127] The structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model.

[0128] Optionally, since the encrypted data is generated by compression and encryption in the terminal device, the structured data and model-related information corresponding to the encrypted data can be obtained through corresponding decompression and decryption; the decompression method adopted in this embodiment corresponds to the compression processing method in the above embodiment, for example, the above step 3032 is through lossless compression, and this embodiment decompresses the encrypted data through the corresponding decompression method; the decryption method adopted in this implementation corresponds to the encryption processing method in the above embodiment, for example, the above step 3032 is encrypted through a symmetric key, and this embodiment decrypts the encrypted data through a key (the same as the encryption key).

[0129] Step 603: Determine a feature database corresponding to the structured data based on the model-related information.

[0130] The cloud server includes multiple feature databases. The feature data recorded in each feature database can be used for image processing by the corresponding model to obtain facial features. Each feature database corresponds to a different model. Before performing target retrieval through structured data, it is first necessary to determine which feature database to search from. This embodiment determines the feature database through model-related information, where the model-related information may include model type information, etc.

[0131] Step 604: retrieve record data corresponding to the target from the feature database based on the structured data.

[0132] In this embodiment, after the cloud server determines the feature database corresponding to the structured data based on the model-related information, a search is performed in the determined feature database, which narrows the search scope, reduces the use of cloud server resources, saves costs, and improves search efficiency.

[0133] like Figure 7 As shown in the above Figure 6 Based on the illustrated embodiment, step 603 may include the following steps:

[0134] Step 6031: determine the first feature library name, the first model type and the first model version identifier from the model related information.

[0135] In this embodiment, the first feature library name, first model type and first model version identifier obtained from the model-related information correspond to the terminal device that transmits encrypted information, wherein the first feature library name, first model type and first model version identifier are relevant information of the first model for processing the target image in the terminal device.

[0136] Step 6032: determine the service type implemented by the first model in the terminal device based on the first model type, and determine the feature library set corresponding to the structured data according to the service type.

[0137] Each service type corresponds to a feature library set, and the feature library set includes multiple feature databases.

[0138] Service types can be distinguished by corresponding to different targets. For example, when the target is a face, the corresponding service type is face recognition, while when the target is a body, the corresponding service type is body recognition. The feature libraries obtained by processing different versions of the model are different. Therefore, each service type corresponds to multiple feature databases.

[0139] Step 6033: Determine a feature database corresponding to the structured data from the feature library set based on the first feature library name and the first model version identifier.

[0140] In this embodiment, each type of search in the cloud server corresponds to a service, and the message reported by each terminal device is balanced to different search services according to the type; the cloud server creates a feature library (for example Figure 2 In the retrieval target, the face is used as the feature library. In the following, the face is used as the retrieval target) when the model version corresponding to the feature in the feature library is specified, and the model version is specified when searching. The retrieval parameters and thresholds corresponding to different model versions are different; when the terminal device reports data for retrieval, the face library to be retrieved is determined according to the name of the face library associated with the terminal device (each terminal device corresponds to at least one face library) and the model version number information on the terminal device.

[0141] like Figure 8 As shown in the above Figure 7 Based on the illustrated embodiment, step 6033 may include the following steps:

[0142] Step 801 : screening multiple feature databases included in a feature database set based on a first feature database name, and determining at least one feature database whose feature database name matches the first feature database name.

[0143] Step 802: Screen at least one feature database based on the first model version identifier to determine a feature database whose model version identifier corresponds to the first model version identifier.

[0144] Each feature database corresponds to a feature library name and a model version identifier.

[0145] This embodiment implements determining a unique cloud server feature library by adding a feature library name and a model version identifier; the facial features extracted by different models are very different, so to identify a face library, it is necessary not only to specify the name of the face library but also to specify the model version number corresponding to the facial features in the face library. A cloud server face library is uniquely identified by the face library name + model version number. When searching, it is necessary to find the library to be searched based on the face library name and model version number.

[0146] Optionally, the method provided in this embodiment further includes:

[0147] In response to the second model of the terminal device being updated, an updated model version identifier sent by the terminal device is received; and the model version identifier corresponding to the feature database corresponding to the second model is replaced based on the updated model version identifier.

[0148] This embodiment corresponds to the model update step of the terminal device. When the terminal device receives a version update request and performs a version update, the feature database of the model to be updated in the cloud server is identified by the model version to ensure that after the model in the terminal device is updated, the structured data processed by the model can obtain the corresponding feature database in the cloud server.

[0149] In some optional embodiments, the structured data includes target features, and the feature database includes a plurality of record data; wherein each record data includes feature data and feature-related information.

[0150] Optionally, feature-related information may include but is not limited to key points, attributes and other information.

[0151] Step 604 includes: determining at least one record data matching the target feature based on a distance between the target feature and feature data in each of the plurality of record data.

[0152] In this embodiment, by calculating the distance (such as Euclidean distance, cosine distance, etc.) between the target feature and the feature data in each record data, and sorting the multiple record data according to the distance, the record data with the smallest distance is determined. In order to improve the accuracy, optionally, the distance of the record data with the smallest distance is compared with a preset threshold. When the minimum distance is less than the preset threshold, the record data corresponding to the minimum distance is used as the matching record data; or, all calculated distances are compared with the preset threshold, and all record data corresponding to distances less than the preset threshold are used as the matching record data.

[0153] Optionally, step 602 includes:

[0154] Receive the compressed and encrypted data sent by the terminal device; perform decryption and decompression on the encrypted data to obtain structured data and model-related information.

[0155] Among them, the decryption process corresponds to the encryption process of the terminal device.

[0156] In this embodiment, the process of processing encrypted data in the cloud server can refer to Figure 2 As shown in step 208 of the provided embodiment, the data reported by the terminal device is decompressed and decrypted to obtain structured data. In order to ensure that the correct structured data and model-related information are obtained, the decryption processing in the cloud server needs to correspond to the encryption processing of the terminal device.

[0157] Any target retrieval method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any target retrieval method provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any target retrieval method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0158] Exemplary Devices

[0159] Fig. 9 FIG. 1 is a schematic diagram of the structure of a terminal device provided by an exemplary embodiment of the present disclosure. Fig. 9 As shown, the terminal device provided in this embodiment includes:

[0160] The image processing module 91 is used to process the collected target image through the first model to obtain structured data.

[0161] The information determination module 92 is used to determine the model-related information corresponding to the first model provided by the image processing module 91 .

[0162] The data generating module 93 is used to generate encrypted data based on the structured data obtained by the image processing module 91 and the model related information determined by the information determining module 92 .

[0163] The data sending module 94 is used to send the encrypted data obtained by the data generating module 93 to the cloud server.

[0164] The above-mentioned embodiment of the present disclosure provides a terminal device, which processes the collected target image through a first model to obtain structured data; determines the model-related information corresponding to the first model; generates encrypted data based on the structured data and the model-related information; and sends the encrypted data to a cloud server. Since the structured data rather than the original snapshot or video stream is reported to the cloud server in this embodiment, the bandwidth occupancy is reduced, the use of cloud server resources is reduced, the cost is saved, and the retrieval efficiency is improved. Moreover, since the structured data does not include specific target information, the security and privacy of the target data are guaranteed.

[0165] Fig.10 FIG. 1 is a schematic diagram of the structure of a terminal device provided by another exemplary embodiment of the present disclosure. Fig.10 As shown, the terminal device of this embodiment includes:

[0166] In this embodiment, the image processing module 91 includes: a model determination unit 911, which is used to determine a first model from at least one model included in the terminal device; wherein at least one model corresponds to at least one model type, and models of different model types perform different processing on the target image;

[0167] The feature extraction unit 912 is used to extract features from the target image using the determined first model to obtain target features; and use the target features as structured data.

[0168] The information determination module 92 includes: a model information determination unit 921, which is used to determine the feature library name, model type and model version identifier of the first model;

[0169] The related information determining unit 922 is used to determine the model related information corresponding to the first model based on the feature library name, model type and model version identifier.

[0170] The data generation module 93 includes: an encapsulation unit 931, which is used to encapsulate the structured data, the feature library name, the model type and the model version identifier according to a set format to obtain encapsulated data with the set format; wherein the set format corresponds to the recognition format of the cloud server;

[0171] The processing unit 932 is used to compress and encrypt the encapsulated data to obtain processed encrypted data.

[0172] Optionally, the terminal device provided in this embodiment further includes:

[0173] The image acquisition module 95 is used to acquire an image or video including a target; perform target area detection and quality screening on at least one frame of video image in the image or video to obtain a target image.

[0174] The version update module 96 is used to respond to the detection of a model version update request and update the corresponding second model according to the model version update request; save the updated model version identifier and send the updated model version identifier to the cloud server to update the feature database in the cloud server.

[0175] Fig.11 FIG. 1 is a schematic diagram of the structure of a cloud server provided by an exemplary embodiment of the present disclosure. Fig.11 As shown, the cloud server provided in this embodiment includes:

[0176] The data receiving module 111 is used to receive encrypted data sent by the device.

[0177] The data parsing module 112 is used to determine structured data and model-related information based on the encrypted data received by the data receiving module 111 .

[0178] The structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model.

[0179] The database search module 113 is used to determine the feature database corresponding to the structured data based on the model related information determined by the data analysis module 112.

[0180] The data matching module 114 is used to retrieve the record data corresponding to the target from the feature database determined by the database search module 113 based on the structured data determined by the data analysis module 112 .

[0181] In this embodiment, after the cloud server determines the feature database corresponding to the structured data based on the model-related information, a search is performed in the determined feature database, which narrows the search scope, reduces the use of cloud server resources, saves costs, and improves search efficiency.

[0182] Fig.12 FIG. 1 is a schematic diagram of the structure of a cloud server provided by another exemplary embodiment of the present disclosure. Fig.12 As shown, the cloud server provided in this embodiment includes:

[0183] In this embodiment, the data parsing module 112 is specifically used to receive encrypted data that has been compressed and encrypted and sent by a terminal device; perform decryption and decompression processing on the encrypted data to obtain structured data and model-related information; wherein the decryption processing corresponds to the encryption processing of the terminal device.

[0184] The database search module 113 includes a model identification unit 1131, which determines a first feature library name, a first model type, and a first model version identifier from model related information;

[0185] A library search unit 1132 is used to determine the service type implemented by the first model in the terminal device based on the first model type, and determine a feature library set corresponding to the structured data according to the service type; wherein each service type corresponds to a feature library set, and the feature library set includes multiple feature databases;

[0186] The library corresponding unit 1133 is used to determine the feature database corresponding to the structured data from the feature library set based on the first feature library name and the first model version identifier.

[0187] Optionally, the library corresponding unit 1133 is specifically used to screen multiple feature databases included in the feature library set based on the first feature library name, and determine at least one feature database whose feature library name matches the first feature library name; screen at least one feature database based on the first model version identifier, and determine a feature database whose model version identifier corresponds to the first model version identifier; wherein each feature database corresponds to a feature library name and a model version identifier.

[0188] Optionally, the structured data includes a target feature, and the feature database includes multiple record data; wherein each record data includes feature data and feature-related information; the data matching module 114 is specifically used to determine at least one record data that matches the target feature based on the distance between the target feature and the feature data in each of the multiple record data.

[0189] Optionally, the cloud server provided in this embodiment further includes:

[0190] The identification updating module 115 is used to perform version updating in response to the second model of the terminal device, receive the updated model version identification sent by the terminal device, and replace the model version identification corresponding to the feature database corresponding to the second model based on the updated model version identification.

[0191] According to another aspect of the embodiments of the present disclosure, a target retrieval system is provided, comprising: a cloud server as provided in any of the above embodiments and at least one terminal device as provided in any of the above embodiments.

[0192] Exemplary Electronic Devices

[0193] Below, reference Fig.13 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device 100 and the second device 200, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0194] Fig.13 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0195] like Fig.13 As shown, the electronic device 130 includes one or more processors 131 and a memory 132 .

[0196] The processor 131 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 130 to perform desired functions.

[0197] The memory 132 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 131 may run the program instructions to implement the target retrieval method of each embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0198] In one example, the electronic device 130 may further include: an input device 133 and an output device 134 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0199] For example, when the electronic device is the first device 100 or the second device 200, the input device 133 may be the microphone or microphone array described above, for capturing input signals of a sound source. When the electronic device is a stand-alone device, the input device 133 may be a communication network connector, for receiving collected input signals from the first device 100 and the second device 200.

[0200] In addition, the input device 133 may also include, for example, a keyboard, a mouse, and the like.

[0201] The output device 134 can output various information to the outside, including determined distance information, direction information, etc. The output device 134 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0202] Of course, to simplify, Fig.13 Only some of the components related to the present disclosure in the electronic device 130 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 130 may also include any other appropriate components.

[0203] Exemplary computer program products and computer-readable storage media

[0204] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the target retrieval method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0205] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0206] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the target retrieval method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0207] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0208] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0209] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0210] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0211] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0212] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0213] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0214] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A target retrieval method, applied to a terminal device, comprising: Processing the acquired target image through the first model to obtain structured data; The structured data includes target features, target attributes and key point information; Determine model-related information corresponding to the first model; Generate encrypted data based on the structured data and the model-related information; Sending the encrypted data to a cloud server; The determining model-related information corresponding to the first model includes: Determine a feature library name, a model type, and a model version identifier of the first model; determine model-related information corresponding to the first model based on the feature library name, the model type, and the model version identifier; The invention also includes: in response to detecting a model version update request, performing a version update on a corresponding second model according to the model version update request; The updated model version identifier is saved, and the updated model version identifier is sent to a cloud server to update the feature database in the cloud server.

2. The method according to claim 1, wherein generating encrypted data based on the structured data and the model-related information comprises: Encapsulating the structured data, the feature library name, the model type and the model version identifier in a set format to obtain encapsulated data in the set format; The setting format corresponds to the recognition format of the cloud server; The encapsulated data is compressed and encrypted to obtain processed encrypted data.

3. The method according to claim 2, wherein: The step of processing the collected target image by using the first model to obtain structured data includes: Determine a first model from at least one model included in the terminal device; wherein the at least one model corresponds to at least one model type, and the models of different model types perform different processing on the target image; The target image is subjected to feature extraction through the determined first model to obtain target features; and the target features are used as the structured data.

4. The method according to any one of claims 1 to 3, before processing the acquired target image by the first model to obtain structured data, further comprising: Acquiring images or videos including the target; Perform target area detection and quality screening on the image or at least one frame of video image in the video to obtain a target image.

5. A target retrieval method, applied to a cloud server, comprising: Receiving encrypted data sent by a terminal device; Based on the encrypted data, determine structured data and model-related information; wherein the structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model; the model-related information includes a first feature library name, a first model type, and a first model version identifier; Determine a feature database corresponding to the structured data based on the model related information; Retrieving the record data corresponding to the target from the feature database based on the structured data; Also includes: In response to the second model of the terminal device being updated, receiving an updated model version identifier sent by the terminal device; The model version identifier corresponding to the feature database corresponding to the second model is replaced based on the updated model version identifier.

6. The method according to claim 5, wherein: The determining, based on the model-related information, a feature database corresponding to the structured data includes: Determine a first feature library name, a first model type, and a first model version identifier from the model-related information; Determine the service type implemented by the first model in the terminal device based on the first model type, and determine the feature library set corresponding to the structured data according to the service type; wherein each service type corresponds to a feature library set, and the feature library set includes a plurality of feature databases; A feature database corresponding to the structured data is determined from the feature library set based on the first feature library name and the first model version identifier.

7. The method according to claim 6, wherein: The determining, from the feature library set based on the first feature library name and the first model version identifier, a feature database corresponding to the structured data includes: Screening a plurality of feature databases included in the feature database set based on the first feature database name, and determining at least one feature database whose feature database name matches the first feature database name; The at least one feature database is screened based on the first model version identifier to determine a feature database whose model version identifier corresponds to the first model version identifier; wherein each feature database corresponds to a feature database name and a model version identifier.

8. The method according to any one of claims 5 to 7, wherein: The structured data includes target features, and the feature database includes a plurality of record data; wherein each of the record data includes feature data and feature-related information; The retrieving the record data corresponding to the target from the feature database based on the structured data includes: At least one record data matching the target feature is determined based on a distance between the target feature and feature data in each of the plurality of record data.

9. The method according to claim 5, wherein: The receiving terminal device sends structured data and model-related information, including: Receiving the compressed and encrypted encrypted data sent by the terminal device; The encrypted data is decrypted and decompressed to obtain the structured data and the model-related information; wherein the decryption process corresponds to the encryption process of the terminal device.

10. A terminal device, comprising: An image processing module, used to process the collected target image through the first model to obtain structured data; An information determination module, used to determine model-related information corresponding to the first model provided by the image processing module; a data generating module, configured to generate encrypted data based on the structured data obtained by the image processing module and the model-related information determined by the information determining module; A data sending module, used to send the encrypted data obtained by the data generating module to a cloud server; The information determination module includes: a model information determination unit, which is used to determine the feature library name, model type and model version identifier of the first model; The related information determining unit is used to determine the model related information corresponding to the first model based on the feature library name, model type and model version identifier.

11. A cloud server, comprising: A data receiving module, used for receiving encrypted data sent by a terminal device; A data parsing module, configured to determine structured data and model-related information based on the encrypted data received by the data receiving module; wherein the structured data is obtained by processing the target image corresponding to the first model, and the model-related information corresponds to the first model; and the model-related information includes a first feature library name, a first model type, and a first model version identifier; A database search module, used to determine a feature database corresponding to the structured data based on the model related information determined by the data analysis module; The data matching module is used to retrieve the record data corresponding to the target from the feature database determined by the database search module based on the structured data determined by the data parsing module.

12. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the target retrieval method according to any one of claims 1 to 9.

13. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the target retrieval method described in any one of claims 1-9 above.

Images