Risk identification method and device, storage medium and electronic equipment
By comparing the target detection model based on open set detection and the risk feature library, the problems of inaccurate risk identification and long iteration cycle in the existing technology are solved, and high-accuracy risk identification is achieved.
Patent Information
- Application Number
- CN202411269690.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2025-01-21
AI Technical Summary
When prior art recognizes whether there is a risk in images transmitted on the Internet, the model iteration cycle is long, and the text output by the model cannot accurately describe the target objects in the image and the risks present in the target objects, resulting in inaccurate risk identification.
The target detection model based on open set detection is used to determine the risk identification result of the image by extracting the image features of the target object in the image to be identified and comparing it with the preset risk feature library.
This method can effectively improve the accuracy of risk identification. It does not require the model to iteratively learn new risks, and only needs to update the risk feature database to identify new risks.
Smart Images
Figure CN119339109A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a risk identification method, device, storage medium and electronic device. Background Art
[0002] With the rapid development of the Internet, content risks have attracted widespread attention. Among them, identifying whether there are risks in images spread by users on the Internet and conducting risk control on the images has become a key issue.
[0003] Generally, a machine learning model is used to identify whether there is a risk in an image. Specifically, the image to be identified is input into the machine learning model, and each target object in the image is identified by the machine learning model, and then it is determined whether each target object is risky, that is, whether there is a risk target object that is not allowed to be transmitted on the network among each target object, and finally the identification result is output in text. The output text is used not only to describe whether the image is risky, but also to describe each target object in the image, whether each target object is a risk target object, what kind of risk it has, etc.
[0004] However, the above method of the prior art requires pre-definition of various risks and then training the model. When new risks appear, the iteration cycle of the model is too long. Moreover, in practical applications, the text output by the model cannot accurately describe the target objects in the image and the risks existing in the target objects. Therefore, the above model cannot accurately identify risks. Summary of the invention
[0005] The embodiments of this specification provide a Bluetooth-based distance measurement method, device, storage medium, and electronic device to partially solve the problems existing in the above-mentioned prior art.
[0006] The embodiments of this specification adopt the following technical solutions:
[0007] This specification provides a risk identification method, the method comprising:
[0008] Obtain an image to be recognized;
[0009] Inputting the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified;
[0010] Extracting image features of each target object from the image to be identified;
[0011] The image features of each target object are compared with the risk features stored in a preset risk feature library, and the risk identification result of the image to be identified is determined according to the comparison result.
[0012] Optionally, the target detection model includes: a Grounding DINO model.
[0013] Optionally, inputting the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified specifically includes:
[0014] The image to be identified is input into a pre-trained target detection model based on open set detection, so that the target detection model determines the outer bounding box of each target object in the image to be identified and outputs it.
[0015] Optionally, extracting image features of each target object from the image to be identified specifically includes:
[0016] Extracting full-image features of the image to be identified;
[0017] For each target object, the image features of the target object are determined according to the position of the outer bounding box of the target object in the image to be identified and the full image features.
[0018] Optionally, extracting full-image features of the image to be identified specifically includes:
[0019] The full-image features of the image to be identified are extracted through the DINOV2 representation model.
[0020] Optionally, a risk feature library is preset, specifically including:
[0021] Acquire a sample image containing a risk target;
[0022] The image features of the risk target objects in the sample images are extracted and stored in a risk feature library as risk features.
[0023] Optionally, determining a risk identification result of the image to be identified according to the comparison result specifically includes:
[0024] When there is at least one target object whose image feature has a similarity with the risk feature greater than a preset threshold, it is determined that the image to be identified contains a risk target object.
[0025] Optionally, the risk feature library also stores security features;
[0026] The image features of each target object are compared with the risk features stored in a preset risk feature library, and the risk identification result of the image to be identified is determined according to the comparison result, specifically including:
[0027] For each image feature of the target object, determining the similarity between the image feature of the target object and each risk feature stored in the risk feature library;
[0028] In order of similarity between the image feature of the target object and each risk feature, a first specified number of risk features are selected from the risk features and added to the feature set; and a second specified number of safety features are selected from the risk feature library and added to the feature set;
[0029] For each risk feature in the feature set, sorting the features in the feature set in descending order of similarity between the risk feature and other features in the feature set;
[0030] The risk feature, other features in the feature set and the ranking result are input into a pre-trained recognition model to identify whether the target object has the risk corresponding to the risk feature through the recognition model.
[0031] This specification provides a risk identification device, the device comprising:
[0032] An acquisition module, used for acquiring an image to be recognized;
[0033] A target detection module, used for inputting the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified;
[0034] A feature extraction module, used to extract the image features of each target object from the image to be identified;
[0035] The recognition module is used to compare the image features of each target object with the risk features stored in a preset risk feature library, and determine the risk recognition result of the image to be recognized based on the comparison result.
[0036] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the risk identification method described above is implemented.
[0037] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the risk identification method when executing the program.
[0038] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:
[0039] The embodiments of this specification disclose a risk identification method. This method retains the target detection function of the machine learning model, abandons the text description function of the machine learning model, uses machine learning to identify the target object from the image to be identified, and then extracts the image features of each target object. The risk features extracted and saved in advance are compared with the image features of each target object to identify whether there is a risk in the image to be identified. There is no need for the machine learning model to iteratively learn new risks, which can effectively improve the accuracy of risk identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:
[0041] Figure 1 A schematic diagram of a risk identification process provided in an embodiment of this specification;
[0042] Figure 2 A schematic diagram of the structure of a risk identification device provided in an embodiment of this specification;
[0043] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.
[0045] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.
[0046] Figure 1 The following is a flow chart of risk identification in this specification, which may include the following steps:
[0047] S100: Acquire an image to be recognized.
[0048] The risk identification method provided in this manual can be used to identify whether images generated or disseminated by users on the Internet are risky. The devices used to identify whether images are risky may include any electronic devices such as terminals (such as computers, mobile phones, etc.) and servers, and this manual does not limit this.
[0049] When acquiring images to be identified, images generated or spread by users on the Internet can be acquired as images to be identified. If the server serves as the executor of risk identification, the image can be acquired as the image to be identified when the user uploads the image to the server through his terminal. If the terminal serves as the executor of risk identification, the image can be acquired as the image to be identified when the user generates or downloads the image.
[0050] The following is an example of using a server as the executor of risk identification.
[0051] S102: Input the image to be identified into a pre-trained open set detection-based target detection model to obtain each target object detected by the target detection model from the image to be identified.
[0052] In an embodiment of the present specification, an open set detection-based target detection model can be pre-trained, and the target detection model is used to identify each target object contained in the image input to the model. Among them, open set detection is a task in machine learning, which involves identifying samples of known categories and refusing to identify samples of unknown categories. In traditional supervised learning, machine learning models are trained to distinguish several predefined categories, and when using the model, it is necessary to assume that the data to be identified will come from one of these known categories. However, in practical applications, the data to be identified may contain samples that do not belong to any category in the training data. The open set detection provides at least one other category other than the categories predefined during training. When the data to be identified does not belong to any of the categories predefined during training, the model can output the above other categories as the recognition result. For example, a model using supervised training is used to identify whether the animal in the image is a cat or a dog, then cats and dogs are two categories predefined during training, and the model based on open set detection will provide other categories besides cats and dogs. The other category can be named "unknown category", for example. If the model based on open set detection is used, the input model is an image of a bear, then the output result of the model is neither a cat nor a dog, but an unknown category. In short, the purpose of open set detection is to recognize that an unknown category is unknown when it is encountered, rather than forcibly classifying it into a known category predefined during training.
[0053] Then in step S102, after the image to be identified is input into the object detection model based on open set detection, the object detection model can output each object detected from the image to be identified. For an object in the image to be identified, even if the object detection model does not know what kind of object the object is, it can still know that it is an unknown class of object based on open set detection and output it.
[0054] S104: Extracting image features of each target object from the image to be identified.
[0055] After detecting each target object from the image to be identified by the target detection model, the image features of each target object can be extracted from the image to be identified. The image features of the target object may include any interpretable image features of the target object in the image to be identified, and also include any image features that are not interpretable or are weakly interpretable. Interpretable image features may include color features, texture features, edge features, etc. These interpretable image features can be extracted by any image processing algorithm, such as using Hough transform to extract edge features. Image features that are not interpretable or are weakly interpretable can be extracted by a pre-trained feature extraction model, and the extracted features can be latent features in the form of vectors or matrices.
[0056] S106: Compare the image features of each target object with the risk features stored in a preset risk feature library, and determine the risk identification result of the image to be identified according to the comparison result.
[0057] In the embodiment of the present specification, it is necessary to maintain a risk feature library, which stores the image features of risky images, namely, risk features. After extracting the image features of each target object in the image to be identified through step S104, the image features of each target object can be compared with all the risk features in the risk feature library to determine the similarity between the image features of each target object and each risk feature. When there is at least one target object whose image feature has a similarity with any risk feature in the risk feature library greater than a preset threshold, it can be determined that the image to be identified contains a risk target object, and the risk target object is a target object whose similarity with any risk feature in the risk feature library is greater than the preset threshold, and then a predetermined risk control strategy can be used to perform risk control on the image to be identified. If there is no target object whose image feature has a similarity with any risk feature in the risk feature library greater than the preset threshold, it is determined that the image to be identified is risk-free. The similarity between the image feature of the target object and the risk feature can be characterized by methods such as Euclidean distance or cosine distance.
[0058] It can be seen from the above method that since the text output by the machine learning model usually cannot accurately describe the target objects in the image and the risks existing in the target objects, the risk identification method provided in this specification directly abandons the text output capability of the model, while retaining the target detection capability of the model, and improving it to a target detection model based on open set detection, so that even if it is unknown what target object is contained in the image to be identified, it can still detect and output the target object, and then extract the image features of each target object, and compare them with the risk features in the maintained risk feature library, and identify whether the image to be identified is risky based on the comparison results.
[0059] Since the model is only used to detect target objects, there is no need to iterate the model when a new risk appears. It is only necessary to extract the image features of the new risk and store them in the risk feature library as new risk features to identify the new risk. Therefore, the new risk can also be identified immediately and effectively.
[0060] Furthermore, the above-mentioned open set detection-based target detection model may include a Grounding DINO model. The Grounding DINO model includes two encoders and a decoder, the two encoders are an image backbone network for extracting image features and a text backbone network for extracting text features, the model also includes a feature enhancement layer for fusing image features and text features and a language prompt retrieval module for retrieval initialization, and the decoder is a cross-mode decoder for predicting the outer bounding box of each target object in the image to be identified.
[0061] In step S102, the image to be identified can be input into the Grounding DINO model, so that the Grounding DINO model determines the outer bounding box of each target object in the image to be identified and outputs it. It should be noted that although the Grounding DINO model can also output text descriptions of the target objects at the same time, these text descriptions are meaningless for the risk identification provided in this specification and can be directly discarded.
[0062] After obtaining the image to be identified with the outer bounding box of each target object, in step S104, the full-image feature of the image to be identified can be extracted first, and then for each target object, the image feature of the target object can be determined according to the position of the outer bounding box of the target object in the image to be identified and the full-image feature. Specifically, after obtaining the full-image feature of the image to be identified, for each target object, the mask corresponding to the target object can be determined according to the position of the outer bounding box of the target object in the image to be identified, and the image feature of the target object can be extracted from the full-image feature using the mask corresponding to the target object.
[0063] Of course, when extracting the image features of each target object, it is also possible to first segment the sub-images corresponding to each target object from the image to be identified according to the outer bounding box of each target object, and then extract the image features corresponding to the target object from each sub-image.
[0064] Whether extracting full-image features from the image to be identified first, or extracting image features corresponding to the target object from the sub-images corresponding to each target object, the feature extraction model can be used to extract. In an optional embodiment of the present specification, the full-image features of the image to be identified can be extracted by the DINOV2 characterization model, or the image features corresponding to the target object can be extracted from the sub-images corresponding to each target object by the DINOV2 characterization model. Of course, those skilled in the art will appreciate that the above features can also be extracted by models such as Clip, which are designed to learn general visual representations of images in an unsupervised or self-supervised manner through the learning of large-scale data sets.
[0065] After extracting the image features of each target object, each target object can be compared with each risk feature in the risk feature library through step S106 to determine the risk identification result of the image to be identified. Specifically, when establishing and maintaining the risk feature library, a sample image containing risk targets can be obtained, and the image features of the risk targets in the sample images can be extracted and stored as risk features in the risk feature library. It should be noted that in the embodiment of this specification, the method of extracting the image features of the target object from the image to be identified in step S104 must be consistent with the method of extracting the image features of the risk target when establishing and maintaining the risk feature library.
[0066] In addition, in order to further improve the accuracy of risk identification, the risk feature library in the embodiment of this specification may also store safety features, and the safety features are image features of images that do not pose risks. Figure 1 In the step S106 shown, when comparing the image features of each target object with the risk features stored in the risk feature library, the similarity between the image features of each target object and the risk features stored in the risk feature library can also be determined for each image feature of the target object; a first specified number of risk features are screened from the risk features in order of similarity from high to low, and the risk features are added to a feature set; and a second specified number of security features are screened from the risk feature library and added to the feature set; for each risk feature in the feature set, the features in the feature set are sorted in order of similarity from high to low with respect to the risk feature and other features in the feature set; the risk feature, other features in the feature set and the sorting result are input into a pre-trained recognition model to identify whether the target object has the risk corresponding to the risk feature through the recognition model.
[0067] For example, for an image feature f of a target object, we can select k (i.e., the first specified number) risk features c with the highest similarity according to the similarity between the image feature f and each risk feature c, and then select n (i.e., the second specified number) security features b from each security feature b, and add the k risk features c and the n security features b to the feature set. Then, for the i-th risk feature c among the k risk features c, i , determine the risk characteristic c i The similarity with all other features in the feature set (including other risk features c in the feature set and all safety features b in the feature set) is then sorted in order from high to low according to the determined similarity. Finally, the binary classification recognition model is used to output whether the image feature of the target object has c. i The corresponding risks.
[0068] The purpose of using the above method is to consider both the characteristics of risk features and the characteristics of security features. First, based on the similarity between the image features of the target object and the risk features, the most suspicious k risk features are selected, and then the k risk features are scrambled using security features. This allows the subsequent recognition model to use the similarity between the k risk features and the scrambled security features as a reference to determine whether the image features of the target image are the k risk features.
[0069] Of course, in order to further improve the accuracy of risk identification, when screening the second specified number of security features to be added to the feature set, n (i.e., the second specified number) security features with the highest similarity to the image features of the target object can also be selected to be added to the feature set.
[0070] The above is a risk identification method provided in an embodiment of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.
[0071] Figure 2 A schematic diagram of a risk identification device provided in an embodiment of this specification, the device includes:
[0072] An acquisition module 201 is used to acquire an image to be recognized;
[0073] The target detection module 202 is used to input the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified;
[0074] A feature extraction module 203 is used to extract the image features of each target object from the image to be identified;
[0075] The identification module 204 is used to compare the image features of each target object with the risk features stored in a preset risk feature library, and determine the risk identification result of the image to be identified according to the comparison result.
[0076] Optionally, the target detection model includes: a Grounding DINO model.
[0077] Optionally, the target detection module 202 is specifically configured to input the image to be identified into a pre-trained target detection model based on open set detection, so that the target detection model determines the outer bounding box of each target object in the image to be identified and outputs it.
[0078] Optionally, the feature extraction module 203 is specifically used to extract the full-image features of the image to be identified; for each target object, the image features of the target object are determined according to the position of the outer bounding box of the target object in the image to be identified and the full-image features.
[0079] Optionally, the feature extraction module 203 is specifically configured to extract full-image features of the image to be identified through a DINOV2 characterization model.
[0080] Optionally, the recognition module 204 is further configured to obtain a sample image containing a risk target object; extract image features of the risk target object in the sample image, and store the image features as risk features in a risk feature library.
[0081] Optionally, the recognition module 204 is specifically configured to determine that the image to be recognized contains a risk target when there is at least one image feature of the target object whose similarity to the risk feature is greater than a preset threshold.
[0082] Optionally, the risk feature library also stores security features;
[0083] The identification module 204 is specifically used to determine the similarity between the image feature of each target object and each risk feature stored in the risk feature library;
[0084] In order of similarity between the image feature of the target object and each risk feature, a first specified number of risk features are selected from the risk features and added to the feature set; and a second specified number of safety features are selected from the risk feature library and added to the feature set;
[0085] For each risk feature in the feature set, sorting the features in the feature set in descending order of similarity between the risk feature and other features in the feature set;
[0086] The risk feature, other features in the feature set and the ranking result are input into a pre-trained recognition model to identify whether the target object has the risk corresponding to the risk feature through the recognition model.
[0087] This specification also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, it can be used to perform the above Figure 1 Provides risk identification methods.
[0088] based on Figure 1 Based on the Figure 1 The risk identification method shown in the specification also provides Figure 3 The structural diagram of the electronic device shown in FIG. Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The risk identification method described.
[0089] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0090] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0091] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0092] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0093] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0094] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0095] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0096] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0098] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0099] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0100] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0101] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0102] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0103] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0104] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0105] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.
Claims
1. A risk identification method, the method comprising: Obtain an image to be recognized; Inputting the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified; Extracting image features of each target object from the image to be identified; The image features of each target object are compared with the risk features stored in a preset risk feature library, and the risk identification result of the image to be identified is determined according to the comparison result.
2. The method according to claim 1, wherein the target detection model comprises: Grounding DINO model.
3. The method according to claim 2, inputting the image to be identified into a pre-trained target detection model based on open set detection, and obtaining each target object detected by the target detection model from the image to be identified, specifically comprises: The image to be identified is input into a pre-trained target detection model based on open set detection, so that the target detection model determines the outer bounding box of each target object in the image to be identified and outputs it.
4. The method according to claim 3, extracting the image features of each target object from the image to be identified, specifically comprising: Extracting full-image features of the image to be identified; For each target object, the image features of the target object are determined according to the position of the outer bounding box of the target object in the image to be identified and the full image features.
5. The method according to claim 4, extracting the full-image features of the image to be identified, specifically comprising: The full-image features of the image to be identified are extracted through the DINOV2 representation model.
6. The method according to claim 1, wherein the risk feature library is preset, specifically comprising: Acquire a sample image containing a risk target; The image features of the risk target objects in the sample images are extracted and stored in a risk feature library as risk features.
7. The method according to claim 1, determining the risk identification result of the image to be identified according to the comparison result, specifically comprising: When there is at least one target object whose image feature has a similarity with the risk feature greater than a preset threshold, it is determined that the image to be identified contains a risk target object.
8. The method according to claim 1, wherein the risk feature library also stores security features; The image features of each target object are compared with the risk features stored in a preset risk feature library, and the risk identification result of the image to be identified is determined according to the comparison result, specifically including: For each image feature of the target object, determining the similarity between the image feature of the target object and each risk feature stored in the risk feature library; In order of similarity between the image feature of the target object and each risk feature, a first specified number of risk features are selected from the risk features and added to the feature set; and a second specified number of safety features are selected from the risk feature library and added to the feature set; For each risk feature in the feature set, sorting the features in the feature set in descending order of similarity between the risk feature and other features in the feature set; The risk feature, other features in the feature set and the ranking result are input into a pre-trained recognition model to identify whether the target object has the risk corresponding to the risk feature through the recognition model.
9. A risk identification device, comprising: An acquisition module, used for acquiring an image to be recognized; A target detection module, used for inputting the image to be identified into a pre-trained target detection model based on open set detection to obtain each target object detected by the target detection model from the image to be identified; A feature extraction module, used to extract the image features of each target object from the image to be identified; The recognition module is used to compare the image features of each target object with the risk features stored in a preset risk feature library, and determine the risk recognition result of the image to be recognized based on the comparison result.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 8.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.