Method and device for identifying biological objects

Abstract

The present invention discloses a biological object recognition method and device. The method includes: recognizing multiple biological feature information and multiple identification information of the biological object from a video; obtaining multiple information combinations of the biological object based on the multiple biological feature information and multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; determining the target biological object based on the multiple categories of biological information, thereby solving the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for identifying a biological object. Background Art

[0002] Video character recognition technology is a popular computer technology that can be used to identify the identities of people in videos. The basic principle of this technology is to match the character feature information extracted from the video with the feature information of existing characters in the base database, and use the character identity information corresponding to the successfully matched feature information in the base database as the identity information of the character in the video. The implementation of this technology depends on the information of existing characters in the base database. For the characters to be identified that exist in the base database and successfully match the feature information, accurate character identity information can be given. For the characters to be identified that do not exist in the base database or whose feature information similarity does not meet the conditions and leads to matching failure, they are often missed in character recognition. In order to identify more characters in the video, it is necessary to continuously expand the character information in the base database. The general solution for expanding the character information in the base database in the existing technology center is to manually screen and annotate the characters to be expanded from the video. This solution has problems such as long cycle, difficult screening, and high annotation cost, which seriously limits the application of video character recognition technology.

[0003] With regard to the problem of low efficiency in biological object recognition due to manual screening and labeling of feature information in the prior art, no effective solution has been proposed so far. Summary of the invention

[0004] The embodiments of the present invention provide a biological object recognition method and device to at least solve the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information.

[0005] According to one aspect of an embodiment of the present invention, a method for identifying a biological object is provided, comprising: identifying multiple biometric information and multiple identification information of the biological object from a video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining a target biological object based on the multiple categories of biological information.

[0006] Furthermore, identifying multiple biometric information and multiple identification information of a biological object from a video includes: extracting audio and text information in the video to obtain video text information; and identifying the video text information according to a preset naming rule to obtain multiple naming information.

[0007] Furthermore, after identifying multiple biometric information and multiple identification information of a biological object from a video, the method includes: setting a unique ID for each of the multiple biometric information; calculating the similarity between each of the biometric information; and associating the biometric information whose similarity meets a preset condition according to the ID to obtain the multiple biometric information after association processing.

[0008] Furthermore, the multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. Based on the multiple biometric information and the multiple identification information, multiple information combinations of biological objects are obtained, including: calculating the time difference between the start and end time of each biometric information and the start and end time of each identification information; associating the biometric information and the identification information whose time difference meets preset conditions to obtain multiple personal information combinations.

[0009] Furthermore, the start and end time include a start time and an end time, and associating the biometric information and the identification information whose time difference meets the preset conditions includes: calculating the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; associating the biometric information with the identification information if the difference between the start time and the difference between the end time are both less than a first threshold.

[0010] Furthermore, the multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and clustering the multiple character information combinations includes: matching the identification information corresponding to the associated biometric information; clustering the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0011] Further, based on the multiple categories of biological information, determining the target biological object includes: counting the amount of each category of biological information, and determining the target biological object according to the biological information whose amount is greater than a second threshold.

[0012] According to one aspect of an embodiment of the present invention, a biological object recognition device is provided, the device comprising: a recognition unit, used to recognize multiple biometric information and multiple identification information of the biological object from a video; a first acquisition unit, used to obtain multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; a first processing unit, used to cluster the multiple information combinations of the biological object to obtain multiple categories of biological information; and a first determination unit, used to determine a target biological object based on the multiple categories of biological information.

[0013] Furthermore, the recognition unit also includes: an extraction module, used to extract audio and text information in the video to obtain video text information; and a recognition module, used to recognize the video text information according to a preset naming rule to obtain multiple naming information.

[0014] Furthermore, the device includes: a setting unit, which is used to set a unique ID for each of the multiple biometric information after identifying multiple biometric information and multiple identification information of a biological object from a video; a calculation unit, which is used to calculate the similarity between each of the biometric information; and an association unit, which is used to associate the biometric information whose similarity meets a preset condition according to the ID to obtain the multiple biometric information after association processing.

[0015] According to one aspect of an embodiment of the present invention, a method for identifying biological objects is provided, comprising: receiving a service call request sent by a client, wherein the service call request carries a request for identifying biological object information in a video; responding to the service call request, identifying a target biological object from the video, and outputting an identification result.

[0016] Furthermore, in response to the service call request, identifying the target biological object from the video includes: identifying multiple biometric information and multiple identification information of the biological object from the video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining the target biological object based on the multiple categories of biological information.

[0017] According to one aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned methods for identifying biological objects.

[0018] According to one aspect of an embodiment of the present invention, a processor is provided, wherein the processor is used to run a program, wherein the program executes any one of the above-mentioned biological object recognition methods when running.

[0019] In an embodiment of the present invention, a method of automatically identifying a target biological object from an existing video is adopted, by identifying multiple biometric information and multiple identification information of the biological object from the video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining the target biological object based on the multiple categories of biological information, thereby achieving the purpose of improving the efficiency of biological object recognition, thereby achieving the technical effect of quickly and low-costly identifying the target biological object from the existing video, and further solving the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 is a hardware structure block diagram of a computer terminal according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a biological object recognition method provided in accordance with the first embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of clustering biological information provided according to the first embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of an optional biological object recognition method provided according to the first embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a biological object recognition device provided according to a second embodiment of the present invention;

[0026] Figure 6 is a flow chart of a biological object recognition method provided in Embodiment 3 of the present invention; and

[0027] Figure 7 is a structural block diagram of an optional computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0031] asr: English: atuomatic speech recongnition, Chinese: automatic speech recognition.

[0032] OCR: English: optical character recognition, Chinese: optical character recognition.

[0033] ner: English: named entity recognition, Chinese: named entity recognition.

[0034] Biometrics: refers to the unique physiological characteristics of each individual that can be measured or automatically identified and verified. It can also be divided into physiological characteristics (such as face, fingerprint, iris, palm print, etc.) and behavioral characteristics (such as gait, voice, handwriting, etc.). Biometric recognition is to identify each individual based on their unique biological characteristics.

[0035] Biological object: an object with kinetic energy, such as a person object.

[0036] Example 1

[0037] According to an embodiment of the present invention, an embodiment of a method for identifying a biological object is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a biological object recognition method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the biological object recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the biological object recognition method of the above-mentioned application program is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, this application provides Figure 2 The biological object recognition method shown. Figure 2 4 is a flow chart of a biological object recognition method according to the first embodiment of the present invention.

[0044] Step S101, identifying multiple biometric information and multiple identification information of a biological object from a video.

[0045] In order to realize the recognition of biological objects from videos, it is necessary to first extract the biometric information and identification information of each biological object from the video, wherein the number of videos may be multiple, and the number of biological objects in the video may also be multiple, and the above-mentioned biological objects may be human objects, the biometric information may be facial information, and the identification information may be naming information. Multiple people in the video may appear in different segments of the video. For each continuous video frame in which a certain human object appears in each segment, only one set of facial feature information is extracted, and the start and end time of the facial feature information appearing in the segment is marked, and the naming information of the person appearing in the segment and the corresponding start and end time are extracted. For example, there are 3 segments in a video in which human objects appear. The facial feature information and naming information of the human object are extracted from each segment, and the start and end time of the facial feature information and naming information in each segment are marked.

[0046] Optionally, in the biological object recognition method provided according to the first embodiment of the present application, recognizing multiple biometric feature information and multiple identification information of the biological object from the video includes: extracting audio and text information in the video to obtain video text information; recognizing the video text information according to preset naming rules to obtain multiple naming information.

[0047] The character naming information in the video may appear in the video voice or subtitles. By processing the data of the voice modality and the subtitle modality, the character naming information in the video is obtained. Among them, the method for obtaining the character naming information from the voice modality data can be: recognizing the voice in the video by ASR technology, processing the recognized result by natural language processing technology, and extracting the character naming information from the result after natural language processing by NER technology and preset rules. The method for obtaining the character naming information from the subtitle modality data can be: recognizing the subtitles by OCR technology, and extracting the character naming information from the text information of the subtitle modality by NER technology combined with preset rules. The data of the voice modality and the subtitle modality in the video are processed separately to obtain the summarized character naming information in the video. For example, the voice and subtitles in 48,855 videos are processed to obtain a total of 100,000 character naming information. For the facial feature information in the video, the facial visual feature information of the characters contained in the video can be obtained by face detection, face tracking, face alignment and feature extraction.

[0048] Optionally, in the biological object recognition method provided according to the first embodiment of the present application, after identifying multiple biometric information and multiple identification information of the biological object from the video, the method includes: setting a unique ID for each biometric information in the multiple biometric information; calculating the similarity between each biometric information; and associating the biometric information whose similarity meets preset conditions according to the ID to obtain multiple biometric information after association processing.

[0049] The facial feature information of the person appearing in each clip of the video is extracted, and a unique ID is set for each extracted facial feature information. Since the biological objects contained in different clips in the video may be repeated, that is, the same person may appear in multiple clips, resulting in a large amount of repeated facial feature information. In order to count the number of times each person object appears in the video, the facial feature information extracted from the same person object in different clips needs to be associated according to the ID number, and multiple biological feature information after ID number association processing is obtained.

[0050] Step S102, obtaining multiple information combinations of the biological object based on multiple biometric feature information and multiple identification information.

[0051] In step S101, the biometric information and identification information of the biological object in the video are extracted, but the correlation between each biometric information and identification information is still uncertain. For a given biometric information, the corresponding identification information cannot be determined. Therefore, it is necessary to associate multiple biometric information and multiple identification information to obtain multiple information combinations of the biological object in the video, each information combination including the associated biometric information and identification information.

[0052] Optionally, in the biological object identification method provided according to the first embodiment of the present application, the multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. Based on the multiple biometric information and the multiple identification information, multiple information combinations of the biological object are obtained, including: calculating the time difference between the start and end time of each biometric information and the start and end time of each identification information; associating the biometric information and identification information whose time difference meets preset conditions to obtain multiple personal information combinations.

[0053] By means of a sliding time window, the biometric information and identification information of each fragment are analyzed and processed separately. When the difference between the start and end time of a fragment of biometric information and the start and end time of a certain identification information is within a preset range, it is considered that the biometric information and the identification information meet the association condition, and the two are associated, that is, the biometric information and the identification information belong to the same biological object.

[0054] Optionally, in the biological object identification method provided according to the first embodiment of the present application, the start and end time include a start time and an end time, and associating the biometric information and identification information whose time difference meets a preset condition includes: calculating the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; associating the biometric information and identification information whose difference between the start time and the difference between the end time are both less than a first threshold.

[0055] When calculating the difference between the start and end time of a segment of biometric information and the start and end time of identification information, since the start and end time include the start time and the end time, it is necessary to calculate the difference between the start time of the biometric information and the start time of the identification information, as well as the difference between the end time of the biometric information and the end time of the identification information. Only when the difference between the start time and the end time of the two are both less than the preset value, can it be considered that the biometric information and the identification information meet the association condition, that is, when the biometric information appears in the video, the identification information appears in the voice or subtitles almost at the same time.

[0056] Step S103, clustering multiple information combinations of biological objects to obtain multiple categories of biological information;

[0057] The same person may appear multiple times in different segments of a video or between different videos. After processing the biological objects in the video according to the above method, multiple repeated biometric information will appear in the extracted biometric information. In order to facilitate the statistics of the number of times each person object appears in the video, it is necessary to cluster multiple information combinations, that is, to classify the biometric information and identification information of the same biological object into one category, thereby obtaining multiple categories of biological information after clustering processing.

[0058] Optionally, in the biological object recognition method provided according to the first embodiment of the present application, multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and clustering multiple personal information combinations includes: matching the identification information corresponding to the associated biometric information; clustering the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0059] In order to improve the accuracy of clustering results, the biometric information and identification information are matched simultaneously, and the data with high similarity of biometric information between different fragments and the corresponding associated identification information are aggregated. The biometric information with high similarity of biometric information and the same corresponding associated identification information is taken as the biometric information of the same category. The biometric information of the same category can determine a biological object. Figure 3 It is a schematic diagram of matching biometric information and identification information, such as Figure 3 As shown, the biometric information and identification information from the three video clips that meet the similarity requirements and the associated identification text information corresponding to each biometric information also matches the same are classified into the same category. For another example, in one use, the amount of associated biometric information data is 182572. The data with high similarity between biometric information of different video clips and the corresponding associated identification information are aggregated by graph node aggregation, and the number of biological objects with similar biometric information and the same associated identification information is 944.

[0060] Step S104, determining the target biological object based on the biological information of multiple categories.

[0061] After clustering the information combination of biological objects, multiple categories of biological information are obtained. The amount of biological information in each category is different. According to the amount of biological information in each category, the target biological object can be determined, and the biological information of the target biological object is added to the base library to expand the biological information of the biological objects in the base library.

[0062] Optionally, in the biological object identification method provided according to the first embodiment of the present application, determining the target biological object based on multiple categories of biological information includes: counting the number of each category of biological information, and determining the target biological object based on the biological information whose number is greater than a second threshold.

[0063] Different biological objects appear in different numbers of times in a video, and the corresponding amount of biological information of each category is also different. In order to identify biological objects with a higher frequency of appearance from the video, and add the characteristic information and corresponding identification information of the biological object to the base library, it is necessary to count the amount of biological information of each category, and take the biological information with a number greater than a preset value as the biological information of the target biological object, thereby determining the target biological object.

[0064] In summary, the biological object recognition method provided in the first embodiment of the present application adopts a method of automatically identifying the target biological object from the existing video, by identifying multiple biometric information and multiple identification information of the biological object from the video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; determining the target biological object based on the multiple categories of biological information, thereby achieving the purpose of improving the efficiency of biological object recognition, thereby achieving the technical effect of quickly and low-costly identifying the target biological object from the existing video, and further solving the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information.

[0065] Figure 4 is a schematic diagram of an optional biological object recognition method provided in accordance with the first embodiment of the present application, such as Figure 4 As shown, by analyzing the video data, the biometric information and identification information of the biological objects in the video are extracted, and the biometric information with high similarity is associated, and then the biometric information and identification information are associated by the sliding time window method, and then the associated biometric information and identification information are clustered, and the biometric information and identification information that meet the conditions and the corresponding associated identification information are classified into the same category, and finally the number of biometric information in each category is counted, and the biometric information and identification information in the category with a number greater than a preset value are taken as the target biological information to be expanded. The above method automatically identifies the target biological objects to be expanded in the video by extracting, aligning, clustering and counting the identification information and biometric information of the biological objects in the video, achieving the purpose of expanding the base database data without manual intervention, effectively solving the problem of difficult screening of artificial base database data and high cost of manual labeling, thereby improving the recognition efficiency of biological objects.

[0066] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0067] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0068] Example 2

[0069] According to an embodiment of the present invention, a device for implementing the above-mentioned biological object recognition method is also provided. Figure 5As shown, the device includes: an identification unit 501, a first acquisition unit 502, a first processing unit 503 and a first determination unit 504.

[0070] Specifically, the recognition unit 501 is used to recognize multiple biometric information and multiple identification information of a biological object from a video;

[0071] A first acquisition unit 502, configured to obtain multiple information combinations of a biological object according to multiple biological characteristic information and multiple identification information;

[0072] A first processing unit 503 is used to cluster multiple information combinations of biological objects to obtain multiple categories of biological information;

[0073] The first determining unit 504 is configured to determine a target biological object based on multiple types of biological information.

[0074] In summary, in the device of the biological object recognition method provided in Example 2 of the present application, multiple biometric feature information and multiple identification information of the biological object are recognized from the video through the recognition unit 501; the first acquisition unit 502 obtains multiple information combinations of the biological object based on the multiple biometric feature information and the multiple identification information; the first processing unit 503 clusters the multiple information combinations of the biological object to obtain multiple categories of biological information; the first determination unit 504 determines the target biological object based on the multiple categories of biological information, thereby achieving the purpose of improving the efficiency of biological object recognition, thereby realizing the technical effect of quickly and low-costly identifying the target biological object from the existing video, and further solving the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information.

[0075] Optionally, in the device of the biological object recognition method provided in Example 2 of the present application, the recognition unit 501 also includes: an extraction module, used to extract audio and text information in the video to obtain video text information; and a recognition module, used to recognize the video text information according to a preset naming rule to obtain multiple naming information.

[0076] Optionally, in the device of the biological object recognition method provided in Example 2 of the present application, the device includes: a setting unit, which is used to set a unique ID for each of the multiple biometric information after identifying multiple biometric information and multiple identification information of the biological object from the video; a calculation unit, which is used to calculate the similarity between each biometric information; and an association unit, which is used to associate the biometric information whose similarity meets the preset conditions according to the ID to obtain multiple biometric information after association processing.

[0077] Optionally, in the device of the biological object recognition method provided in Example 2 of the present application, the multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. The first acquisition unit 502 includes: a calculation module, which is used to calculate the time difference between the start and end time of each biometric information and the start and end time of each identification information; and an association module, which is used to associate the biometric information and identification information whose time difference meets preset conditions to obtain multiple character information combinations.

[0078] Optionally, in the device of the biological object identification method provided in Example 2 of the present application, the start and end time include a start time and an end time, and the association module includes: a calculation submodule, used to calculate the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; an association submodule, used to associate the biometric information and the identification information whose difference between the start time and the difference between the end time are both less than a first threshold.

[0079] Optionally, in the device of the biological object recognition method provided in Example 2 of the present application, the multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and the first processing unit 503 includes: a matching module, used to match the identification information corresponding to the associated biometric information; and a clustering module, used to cluster the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0080] Optionally, in the device of the biological object recognition method provided in Embodiment 2 of the present application, the first determination unit includes: a determination module, which is used to count the number of each category of biological information and determine the target biological object based on the biological information whose number is greater than a second threshold.

[0081] It should be noted that the above identification unit 501 to the first determination unit 504 correspond to steps S101 to S104 in Example 1, and the four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0082] Example 3

[0083] In the operating environment of the above embodiment 1, the present application provides the following Figure 6 The biological object recognition method shown. Figure 6 4 is a flow chart of a biological object recognition method according to Embodiment 3 of the present invention.

[0084] Step S601: receiving a service call request sent by a client, wherein the service call request carries a request for identifying information of biological objects in a video.

[0085] Step S602, responding to the service call request, identifying the target biological object from the video.

[0086] Step S603: output the recognition result.

[0087] By calling service requests to identify the target biological object on the server, the purpose of improving the efficiency of biological object recognition is achieved, thereby achieving the technical effect of quickly and cheaply identifying the target biological object from existing videos, and further solving the technical problem of low efficiency in biological object recognition caused by manual screening and labeling of feature information.

[0088] Optionally, in response to the service call request, identifying the target biological object from the video includes: identifying multiple biometric information and multiple identification information of the biological object from the video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining the target biological object based on the multiple categories of biological information.

[0089] In the above scheme, in order to realize the recognition of biological objects from the video, it is necessary to first extract the biometric information and identification information of each biological object from the video, wherein the number of videos may be multiple, the biological objects in the video may also be multiple, the biological objects may be human objects, the biometric information may be facial information, and the identification information may be naming information. Multiple people in the video may appear in different segments of the video. For the continuous video frames in which a certain human object appears in each segment, only one set of facial feature information is extracted, and the start and end time of the facial feature information appearing in the segment is marked, and the naming information of the person appearing in the segment and the corresponding start and end time are extracted. For example, there are 3 segments in a video in which human objects appear. The facial feature information and naming information of the human object are extracted from each segment, and the start and end time of the facial feature information and naming information in each segment are marked. The biometric information and identification information of the biological objects in the video are extracted, but the correlation between each biometric information and identification information is still uncertain. For a given biometric information, the corresponding identification information cannot be determined. Therefore, it is necessary to associate multiple biometric information and multiple identification information to obtain multiple information combinations of biological objects in the video, each of which includes associated biometric information and identification information. The same person may appear multiple times in different segments of the video or between different videos. After the biological objects in the video are processed according to the above method, multiple repeated biometric information will appear in the extracted biometric information. In order to facilitate the statistics of the number of times each person object appears in the video, it is necessary to cluster the multiple information combinations, that is, to classify the biometric information and identification information of the same biological object into one category, so as to obtain multiple categories of biological information after clustering. After clustering the information combinations of biological objects, multiple categories of biological information are obtained, and the number of biological information of each category is different. According to the number of biological information of each category, the target biological object is determined, and the biological information of the target biological object is added to the base database to expand the biological information of the biological object in the base database.

[0090] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0092] Example 4

[0093] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0094] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0095] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the biological object recognition method of the application: identifying multiple biometric information and multiple identification information of the biological object from the video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining the target biological object based on the multiple categories of biological information.

[0096] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object recognition method of the application: recognizing multiple biometric information and multiple identification information of the biological object from the video includes: extracting the audio and text information in the video to obtain video text information; recognizing the video text information according to the preset naming rules to obtain multiple naming information.

[0097] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object recognition method of the application: after identifying multiple biometric information and multiple identification information of the biological object from the video, the method includes: setting a unique ID for each biometric information in the multiple biometric information; calculating the similarity between each biometric information; associating the biometric information whose similarity meets the preset conditions according to the ID to obtain the multiple biometric information after association processing.

[0098] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object recognition method of the application: the multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. According to the multiple biometric information and the multiple identification information, multiple information combinations of biological objects are obtained, including: calculating the time difference between the start and end time of each biometric information and the start and end time of each identification information; associating the biometric information and the identification information whose time difference meets the preset conditions to obtain multiple character information combinations.

[0099] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object recognition method of the application: the start and end time include the start time and the end time, and the biometric information and the identification information whose time difference meets the preset conditions are associated with each other, including: calculating the difference between the start time of each biometric information and the start time of each identification information, and the difference between the end time of each biometric information and the end time of each identification information; associating the biometric information with the identification information if the difference between the start time and the difference between the end time are both less than a first threshold.

[0100] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object recognition method of the application: the multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and clustering multiple character information combinations includes: matching the identification information corresponding to the associated biometric information; clustering the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0101] The above-mentioned computer terminal can also execute the program code of the following steps in the biological object identification method of the application: based on the multiple categories of biological information, determining the target biological object includes: counting the number of each category of biological information, and determining the target biological object based on the biological information whose number is greater than a second threshold.

[0102] Optionally, Figure 7 is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 7 As shown, the computer terminal may include: one or more ( Figure 7 Only one processor and memory are shown.

[0103] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the biological object recognition method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned biological object recognition method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: identify multiple biometric information and multiple identification information of a biological object from a video; obtain multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; cluster the multiple information combinations of the biological object to obtain multiple categories of biological information; and determine the target biological object based on the multiple categories of biological information.

[0105] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: identifying multiple biometric information and multiple identification information of a biological object from a video, including: extracting audio and text information in the video to obtain video text information; identifying the video text information according to a preset naming rule to obtain multiple naming information.

[0106] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: after identifying multiple biometric information and multiple identification information of a biological object from a video, the method includes: setting a unique ID for each biometric information in the multiple biometric information; calculating the similarity between each biometric information; and associating the biometric information whose similarity meets the preset conditions according to the ID to obtain the multiple biometric information after association processing.

[0107] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: the multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. According to the multiple biometric information and the multiple identification information, multiple information combinations of biological objects are obtained, including: calculating the time difference between the start and end time of each biometric information and the start and end time of each identification information; associating the biometric information and the identification information whose time difference meets the preset conditions to obtain multiple personal information combinations.

[0108] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the start and end time include the start time and the end time, and the biometric information and the identification information whose time difference meets the preset conditions are associated with each other, including: calculating the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; associating the biometric information with the identification information if the difference between the start time and the difference between the end time are both less than a first threshold.

[0109] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: the multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and clustering the multiple character information combinations includes: matching the identification information corresponding to the associated biometric information; clustering the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0110] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: based on the multiple categories of biological information, determining the target biological object includes: counting the number of each category of biological information, and determining the target biological object based on the biological information whose number is greater than a second threshold.

[0111] According to the embodiment of the present invention, a method for identifying biological objects is provided. By automatically identifying the target biological object from an existing video, multiple biological feature information and multiple identification information of the biological object are identified from the video; multiple information combinations of the biological object are obtained based on the multiple biological feature information and the multiple identification information; the multiple information combinations of the biological object are clustered to obtain multiple categories of biological information; based on the multiple categories of biological information, the target biological object is determined, thereby achieving the purpose of improving the efficiency of biological object identification, thereby achieving the technical effect of quickly and low-costly identifying the target biological object from the existing video, and further solving the technical problem of low efficiency of biological object identification caused by manual screening and labeling of feature information.

[0112] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the computer terminal may also be a smart phone (eg, tablet computer, palm computer, mobile Internet device (Mobile Internet Devices, MID), PAD and other terminal devices. Figure 7 The structure of the electronic device is not limited. For example, the computer terminal 10 may also include Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 7 Different configurations are shown.

[0113] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0114] Example 5

[0115] The embodiment of the present invention further provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the biological object recognition method provided in the first embodiment.

[0116] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0117] Optionally, in this embodiment, the storage medium is also configured to store program codes for executing the following steps: identifying multiple biometric information and multiple identification information of a biological object from a video; obtaining multiple information combinations of the biological object based on the multiple biometric information and the multiple identification information; clustering the multiple information combinations of the biological object to obtain multiple categories of biological information; and determining the target biological object based on the multiple categories of biological information.

[0118] Optionally, in this embodiment, the storage medium is also configured to store program codes for executing the following steps: identifying multiple biometric information and multiple identification information of a biological object from a video, including: extracting audio and text information in the video to obtain video text information; identifying the video text information according to a preset naming rule to obtain multiple naming information.

[0119] Optionally, in this embodiment, the storage medium is also configured to store program code for executing the following steps: after identifying multiple biometric information and multiple identification information of a biological object from a video, the method includes: setting a unique ID for each biometric information in the multiple biometric information; calculating the similarity between each biometric information; associating the biometric information whose similarity meets a preset condition according to the ID to obtain the multiple biometric information after association processing.

[0120] Optionally, in this embodiment, the storage medium is also configured to store program code for executing the following steps: the multiple biometric information also includes the start and end time of each biometric information, the multiple identification information also includes the start and end time of each identification information, and based on the multiple biometric information and the multiple identification information, multiple information combinations of biological objects are obtained, including: calculating the time difference between the start and end time of each biometric information and the start and end time of each identification information; associating the biometric information and the identification information whose time difference meets preset conditions to obtain multiple character information combinations.

[0121] Optionally, in this embodiment, the storage medium is also configured to store program code for executing the following steps: the start and end times include a start time and an end time, and associating the biometric information and the identification information whose time difference meets a preset condition includes: calculating the difference between the start time of each biometric information and the start time of each identification information, and the difference between the end time of each biometric information and the end time of each identification information; associating the biometric information with the identification information if the difference between the start time and the difference between the end time are both less than a first threshold.

[0122] Optionally, in this embodiment, the storage medium is also configured to store program code for executing the following steps: the multiple information combinations of the biological object include multiple biometric information and corresponding identification information, and clustering the multiple character information combinations includes: matching the identification information corresponding to the associated biometric information; clustering the biometric information that matches the identification information corresponding to the associated biometric information into the same category of biometric information.

[0123] Optionally, in this embodiment, the storage medium is also configured to store program code for executing the following steps: based on the multiple categories of biological information, determining the target biological object includes: counting the number of each category of biological information, and determining the target biological object based on the biological information whose number is greater than a second threshold.

[0124] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0125] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0130] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying a biological object, characterized in that: The method comprises: recognizing a plurality of biometric information and a plurality of identification information of a biological object from a video; Obtaining multiple information combinations of biological objects according to the multiple biological feature information and the multiple identification information; Clustering multiple information combinations of the biological objects to obtain multiple categories of biological information; Determining a target biological object based on the multiple categories of biological information; The multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. The multiple information combinations of the biological object obtained according to the multiple biometric information and the multiple identification information include: Calculate the time difference between the start and end time of each biometric information and the start and end time of each identification information; The biometric information whose time difference meets the preset condition and the identification information are associated with each other to obtain a plurality of character information combinations.

2. The method according to claim 1, characterized in that The multiple biometric information and multiple identification information for identifying a biological object from a video include: Extracting audio and text information from the video to obtain video text information; The video text information is identified according to a preset naming rule to obtain multiple naming information.

3. The method according to claim 1, characterized in that After identifying a plurality of biometric information and a plurality of identification information of a biological object from a video, the method includes: Setting a unique ID for each of the plurality of biometric information; Calculating the similarity between each biometric feature information; The biometric feature information whose similarity meets the preset condition is associated according to the ID to obtain the multiple biometric feature information after association processing.

4. The method according to claim 1, characterized in that: The start and end time include a start time and an end time, and associating the biometric information and the identification information whose time difference meets a preset condition includes: Calculate the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; The biometric information for which the difference between the start times and the difference between the end times are both smaller than a first threshold is associated with the identification information.

5. The method according to claim 1, characterized in that The plurality of information combinations of the biological object include a plurality of biological feature information and corresponding identification information, and clustering the plurality of person information combinations includes: Matching the identification information corresponding to the associated biometric information; The biometric information that has been associated and has consistent matching identification information corresponding to it is clustered into the same category of biometric information.

6. The method according to claim 1, characterized in that Based on the multiple types of biological information, determining the target biological object includes: The amount of each type of biological information is counted, and the target biological object is determined according to the biological information whose amount is greater than a second threshold.

7. A biological object recognition device, characterized in that: The device comprises: A recognition unit, used to recognize multiple biometric information and multiple identification information of a biological object from a video; A first acquisition unit, configured to obtain a plurality of information combinations of the biological object according to the plurality of biometric feature information and the plurality of identification information; A first processing unit, configured to cluster multiple information combinations of the biological object to obtain multiple categories of biological information; a first determining unit, configured to determine a target biological object based on the plurality of types of biological information; Among them, the multiple biometric information also include the start and end time of each biometric information, and the multiple identification information also include the start and end time of each identification information. The association module includes: a calculation submodule, used to calculate the difference between the start time of each biometric information and each identification information, and the difference between the end time of each biometric information and each identification information; an association submodule, used to associate the biometric information and the identification information whose difference between the start time and the difference between the end time are both less than a first threshold, to obtain multiple character information combinations.

8. The device according to claim 7, characterized in that The identification unit also includes: An extraction module, used to extract audio and text information from the video to obtain video text information; The recognition module is used to recognize the video text information according to a preset naming rule to obtain multiple naming information.

9. The device according to claim 7, characterized in that The device comprises: a setting unit, configured to set a unique ID for each of the plurality of biometric information after recognizing the plurality of biometric information and the plurality of identification information of the biological object from the video; A calculation unit, used for calculating the similarity between each biometric feature information; The associating unit is used to associate the biometric information whose similarity meets the preset conditions according to the ID to obtain the multiple biometric information after association processing.

10. A method for identifying a biological object, characterized in that: include: Receiving a service call request sent by a client, wherein the service call request carries a request for identifying information of a biological object in a video; In response to the service call request, identifying a target biological object from the video; Output recognition results; Wherein, in response to the service call request, identifying the target biological object from the video includes: identifying a plurality of biometric information and a plurality of identification information of a biological object from the video; Obtaining multiple information combinations of biological objects according to the multiple biological feature information and the multiple identification information; Clustering multiple information combinations of the biological objects to obtain multiple categories of biological information; Determining a target biological object based on the multiple categories of biological information; The multiple biometric information also includes the start and end time of each biometric information, and the multiple identification information also includes the start and end time of each identification information. The multiple information combinations of the biological object obtained according to the multiple biometric information and the multiple identification information include: Calculate the time difference between the start and end time of each biometric information and the start and end time of each identification information; The biometric information whose time difference meets the preset condition and the identification information are associated with each other to obtain a plurality of character information combinations.

11. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the biological object recognition method according to any one of claims 1 to 6, or the biological object recognition method according to claim 10.

12. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the biological object recognition method described in any one of claims 1 to 6, or the biological object recognition method described in claim 10.

Images