Target object identification method and system, storage medium and computing device

Abstract

This application discloses a target object identification method and system, storage medium, and computing device. The method comprises: acquiring multiple first images, each of which contains at least one moving object; processing the first images to obtain feature information of the moving object, wherein the feature information includes a first granularity feature and a second granularity feature; and determining the target object based on the feature information of the moving object. This application addresses the technical problem of low accuracy in identifying illegal vehicles in existing target object identification methods.

Description

Technical Field

[0001] The present application relates to the field of traffic monitoring, and more specifically, to a method and system for identifying a target object, a storage medium, and a computing device. Background Art

[0002] In traffic monitoring scenarios, illegal vehicles may use forged or illegally obtained license plate numbers of legal vehicles. Therefore, the vehicle passing records can be recorded by the checkpoint equipment, and based on the time-space correlation, the vehicle passing records with contradictory time-space relationships can be mined to identify illegal vehicles.

[0003] However, the method of identifying illegal vehicles based on spatiotemporal correlation has problems such as inaccurate latitude and longitude calibration of the checkpoint equipment and uncalibrated clocks, which lead to false alarms; the accuracy of the checkpoint equipment in identifying license plate numbers or vehicle attributes is limited, and incorrect identification of license plate numbers or vehicle attributes will lead to false alarms; due to the high false alarm rate, auditors are required to filter and review the machine recognition results; in addition, illegal vehicles often have similar vehicle attributes to legal vehicles, resulting in low recognition accuracy and an inability to accurately identify illegal vehicles.

[0004] Currently, no effective solution has been proposed to the problem that the target object recognition method in the prior art has a low accuracy rate in identifying illegal vehicles. Summary of the Invention

[0005] The embodiments of the present application provide a target object identification method and system, a storage medium, and a computing device to at least solve the technical problem that the target object identification method in the prior art has a low accuracy rate in identifying illegal vehicles.

[0006] According to one aspect of an embodiment of the present application, a method for identifying a target object is provided, including: acquiring multiple captured first images, wherein the first images contain at least one moving object; processing the first images to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining the target object based on the feature information of the moving object.

[0007] According to another aspect of an embodiment of the present application, a method for identifying a target object is also provided, including: acquiring a second image of a moving object to be identified; processing the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature; matching the feature information of the moving object to be identified with feature information of at least one target object stored in a database; if the feature information of the moving object to be identified successfully matches the feature information of any target object stored in the database, determining that the moving object to be identified is a target object.

[0008] According to another aspect of an embodiment of the present application, a storage medium is further provided. The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned target object identification method.

[0009] According to another aspect of an embodiment of the present application, a computing device is further provided, including: a processor and a memory, wherein the processor is configured to run a program stored in the memory, wherein the program executes the above-mentioned target object recognition method when running.

[0010] According to another aspect of an embodiment of the present application, a target object recognition system is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: acquiring multiple captured first images, wherein the first image contains at least one moving object; processing the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining the target object based on the feature information of the moving object.

[0011] According to another aspect of an embodiment of the present application, a method for identifying a target object is also provided, including: acquiring multiple collected first images, wherein the first image contains at least one moving object; processing the first image using a first network model to obtain a first granularity feature of the moving object; processing the first image using a second network model to obtain a second granularity feature of the moving object, wherein the second network model includes: a first sub-network model, a second sub-network model and a third sub-network model, the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network; based on the feature information of the moving object, the target object is determined.

[0012] According to another aspect of an embodiment of the present application, a method for identifying a target object is also provided, including: acquiring multiple captured first images, wherein the first images contain at least one object to be identified; processing the first images to obtain feature information of the object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining the target object based on the feature information of the object to be identified.

[0013] In an embodiment of the present application, after acquiring multiple captured first images, the first images can be processed to obtain feature information such as first and second granularity features of a moving object. Based on the feature information of the moving object, a target object can be determined to achieve the purpose of mobile object recognition. It is readily apparent that because the feature information includes the first and second granularity features, the moving object can be recognized from multiple dimensions, thereby achieving the effect of improving the accuracy and efficiency of mobile object recognition, thereby resolving the technical problem of low accuracy of target object recognition methods in the prior art for identifying illegal vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a target object identification method according to an embodiment of the present application;

[0016] Figure 2 is a schematic diagram of a computer terminal serving as a receiving end according to an embodiment of the present application;

[0017] Figure 3 is a flowchart of a first target object identification method according to an embodiment of the present application;

[0018] Figure 4 is a structural diagram of a vehicle feature extraction model based on an attention mechanism according to an embodiment of the present application;

[0019] Figure 5 This is a flow chart of identifying a duplicate card according to an embodiment of the present application;

[0020] Figure 6 This is a flow chart of real-time vehicle control according to an embodiment of the present application;

[0021] Figure 7 is a flow chart of an optional vehicle identification method according to an embodiment of the present application;

[0022] Figure 8 is a flowchart of a second target object identification method according to an embodiment of the present application;

[0023] Figure 9 is a schematic diagram of a first target object recognition device according to an embodiment of the present application;

[0024] Figure 10is a schematic diagram of a second target object recognition device according to an embodiment of the present application;

[0025] Figure 11 is a flowchart of a third target object identification method according to an embodiment of the present application;

[0026] Figure 12 is a flowchart of a fourth target object identification method according to an embodiment of the present application; and

[0027] Figure 13 This is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0031] Clone license plate vehicle: It may refer to a vehicle in which criminals forge and illegally obtain the license plate number of a legal vehicle, making the illegal vehicle appear legal on the surface.

[0032] Machine Learning: A multidisciplinary field that has emerged over the past 20 years, machine learning encompasses a variety of disciplines, including probability theory, statistics, approximation theory, convex analysis, and computational complexity theory. Machine learning theory primarily focuses on the design and analysis of algorithms that enable computers to automatically "learn."

[0033] Deep Learning: Deep learning is a branch of machine learning that uses algorithms that use complex structures or multiple processing layers composed of multiple nonlinear transformations to perform high-level abstractions on data. Observations (such as an image) can be represented in a variety of ways, such as as a vector of intensity values for each pixel, or more abstractly as a series of edges or regions of a specific shape. Using certain specific representations makes it easier to learn tasks from examples (for example, face recognition or facial expression recognition). The benefit of deep learning is that it replaces manual feature extraction with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

[0034] Computer vision refers to the use of cameras and computers to identify, track, and measure objects, replacing the human eye. This technology then performs image processing, transforming the image into an image more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data.

[0035] Checkpoint equipment: It can refer to equipment on traffic roads or in parks that has monitoring, storage, analysis and other functions, and can obtain images containing vehicles.

[0036] Nearest neighbor algorithm: It can be the simplest method in data mining classification technology. The core idea is that if most of the K nearest neighboring samples of a sample in the feature control belong to a certain category, then the sample also belongs to this category and has the characteristics of the samples in this category.

[0037] Example 1

[0038] According to an embodiment of the present application, a method for identifying a target object is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a target object recognition method. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0040] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the () method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizes the vulnerability detection method of the above-mentioned application. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0042] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0043] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0044] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the aforementioned computer device (or mobile device).

[0045] It should be noted that, in some embodiments, the above Figure 1 The computer device (or mobile device) shown has a touch display (also called a "touch screen" or "touch display screen"). In some embodiments, the above Figure 1 The computer device (or mobile device) shown has a graphical user interface (GUI), and a user can interact with the GUI through finger contacts and / or gestures on the touch-sensitive surface. The human-computer interaction functions here optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interface, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for performing the above-mentioned human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0046] Figure 1 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the computer terminal 10 (or mobile device), but also as an exemplary block diagram of the server. In an optional embodiment, Figure 2 The block diagram shows the use of the above Figure 1 The computer terminal 10 (or mobile device) shown in FIG. 1 is an embodiment of a receiving end. Figure 2As shown, the computer terminal 10 (or mobile device) can be connected to one or more clients 20 via a data network connection or electronic connection. In an optional embodiment, the computer terminal 10 (or mobile device) can be a cloud server. The data network connection can be a local area network connection, a wide area network connection, an Internet connection, or other types of data network connections. The computer terminal 10 (or mobile device) can provide network services for the client 20. Network services are network-based user services such as social networks, cloud resources, email, online payment or other online applications.

[0047] Under the above operating environment, this application provides Figure 3 The target object identification method shown. Figure 3 FIG. 1 is a flow chart of a first target object identification method according to an embodiment of the present application. Figure 3 As shown, the method includes the following steps:

[0048] Step S302: Acquire a plurality of captured first images, wherein each first image contains at least one moving object;

[0049] The moving objects in the above steps can be any movable object in a traffic monitoring scene, such as a vehicle or pedestrian. In the embodiments of this application, a vehicle is used as an example. The first image can be an image containing a moving object captured by a checkpoint device installed on a traffic road, or it can be a surveillance video containing a moving object captured by the checkpoint device. The checkpoint device can capture images at fixed time intervals and can upload the captured images to a server at a fixed time interval according to the capture time, which then performs mobile object recognition.

[0050] Step S304: Process the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature;

[0051] The first granularity feature in the above steps may refer to the coarse-grained feature of the moving object. For example, for a vehicle, the feature may include at least one of the following: color, model and brand, but not limited to these. The second granularity feature may refer to the fine-grained feature of the moving object. For example, for a vehicle, the feature may include at least one of the following: annual inspection mark, decorations and windows, but not limited to these.

[0052] In an optional embodiment, after receiving the first image uploaded by the camera device, the server can identify the received first image based on computer vision technology, deep learning technology, machine learning technology and big data technology, identify the moving object in the image, and further identify the first granularity features and second granularity features of the moving object to obtain multi-dimensional information of the moving object.

[0053] Step S306: Determine the target object based on the feature information of the moving object.

[0054] The target object in the above steps can be a moving object that needs to be identified, for example, it can be an illegal vehicle that needs to be monitored in a traffic monitoring scene, or it can be a vehicle or pedestrian that needs to be identified in a navigation scene, or it can be a vehicle that needs to have its trajectory drawn, but it is not limited to this.

[0055] In an optional embodiment, after identifying the multi-dimensional information of the moving object, the server may perform comparative detection based on the identified feature information to determine whether the identified moving object is the target object, thereby achieving the purpose of mobile object identification.

[0056] For example, using the identification of cloned license plates in traffic monitoring as an example, the camera device can upload the captured first image to the server for processing, or directly upload the captured video (containing multiple images) to the server for processing. After receiving the first image, the server can identify and locate the vehicle contained in the first image, further identify the vehicle's attributes, and extract the vehicle's features to obtain multi-dimensional information about each vehicle. It can then identify cloned license plates by performing feature matching on vehicles with the same license plate number.

[0057] For example, in a navigation scenario, pedestrian recognition can be used. A navigating vehicle can capture a first image in front of it and upload it to a server for processing. After receiving the first image, the server can identify and locate objects within it, extract features of varying granularity, and identify other vehicles in front of it based on these extracted multi-dimensional features. This allows navigation based on these identified vehicles.

[0058] Taking the vehicle trajectory mapping scenario as an example, the camera device can upload the captured first image to the server for processing, or directly upload the captured video (containing multiple images) to the server for processing. After receiving the first image, the server can identify and locate the vehicle contained in the first image, further identify the vehicle's attributes, and extract vehicle features to obtain multi-dimensional information about each vehicle, further identify the target vehicle, and thus map the target vehicle's trajectory.

[0059] Based on the solution provided by the above-mentioned embodiment of the present application, after acquiring multiple first images, the first images can be processed to obtain characteristic information such as the first granularity feature and the second granularity feature of the moving object, and based on the characteristic information of the moving object, the target object can be determined to achieve the purpose of mobile object recognition. It is easy to notice that because the characteristic information includes the first granularity feature and the second granularity feature, the mobile object can be identified in multiple dimensions, thereby achieving the effect of improving the accuracy and efficiency of mobile object recognition, thereby solving the technical problem of the low accuracy of the target object recognition method in the prior art in identifying illegal vehicles.

[0060] In the above embodiment of the present application, the first image is processed to obtain feature information of the moving object, including: processing the first image using a first network model to obtain a first granularity feature of the moving object; processing the first image using a second network model to obtain a second granularity feature of the moving object, wherein the second network model is trained using the first granularity feature.

[0061] The first network model in the above steps can be a trained deep convolutional neural network. This application does not limit the specific structure of the deep convolutional neural network, which can be determined based on actual use requirements and recognition accuracy requirements. The second network model can be a second granularity feature extraction model based on the attention mechanism, including: a first sub-network model, a second sub-network model and a third sub-network model, wherein the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network. Figure 4 As shown, the first sub-network model serves as the backbone trunk network, and the second sub-network model serves as the branch mask network.

[0062] For example, still taking the identification of cloned vehicles in traffic monitoring as an example, in an optional embodiment, after receiving the first image uploaded by the checkpoint device, the server can use a deep convolutional neural network to perform attribute recognition and position calibration on all vehicles in the first image. After determining each vehicle, the first image can be input into the deep convolutional neural network to obtain vehicle features of different scales.

[0063] By using the first network model and the second network model for the first granularity feature and the second granularity feature respectively, the first network model and the second network model can adopt different network structures, and can be set directly based on the recognition accuracy and recognition requirements of the first granularity feature and the second granularity feature. Compared with a single network model, higher recognition accuracy can be obtained.

[0064] In the above embodiment of the present application, the first image is processed using the second network model to obtain the second granularity feature of the moving object, including: using the first sub-network model to process the first image to obtain the first feature of the moving object; using the second sub-network model to process the first image to obtain the second feature of the moving object; combining the first feature and the second feature to obtain a combined feature; using the attention module to process the combined feature to obtain multiple feature channels, wherein different feature channels have different weight values; using the third sub-network model to process the multiple feature channels to obtain the second granularity feature of the moving object.

[0065] In an optional embodiment, as Figure 4 As shown in the figure, after obtaining the vehicle image, the server can use the deep convolutional neural network of the mask branch to process the vehicle image, and obtain vehicle image features of different scales, and then obtain the high-dimensional features of the vehicle (i.e., the first feature mentioned above). At the same time, the deep convolutional neural network of the trunk branch is used to process the vehicle image to obtain the coarse-grained features of the vehicle (i.e., the second feature mentioned above, such as color, brand, and model). The coarse-grained features of the vehicle are used for supervision, and the high-dimensional features and coarse-grained features are combined and the attention module (such as Figure 4 The weighted feature channels are processed using a recurrent neural network to obtain contextual information about the different feature channels. The vehicle features are then obtained by optimizing the loss of the coarse-grained vehicle category and the fine-grained category.

[0066] In the above embodiment of the present application, before using the third network model to process the first image to obtain the second granularity feature of the moving object, the method also includes the following steps: using the third network model to process the first image to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the second network model to process the first image to obtain the second granularity feature of the moving object.

[0067] The third network model in the above step can be a trained deep convolutional neural network. This application does not limit the specific structure of the deep convolutional neural network and can be determined based on actual usage requirements and recognition accuracy requirements. The identification information can be a vehicle license plate number, a pedestrian ID, etc., but is not limited to this.

[0068] It should be noted that the camera device can not only capture images containing moving objects, but can also recognize the captured images, obtain recognition results such as the moving object's identification information, first granularity features, and structured information, and upload the recognition results to the server. In the embodiments of this application, the camera device uploading a license plate number is used as an example. Due to the low recognition accuracy of the camera device, for example, the camera device is prone to misidentification of "0" and "D", "E" and "F" in the license plate number.

[0069] For example, still taking the identification of cloned license plates in traffic monitoring as an example, in an optional embodiment, the server's large-scale computing platform can be used to perform high-precision secondary identification of the license plate number and compare it with the license plate number identified by the checkpoint device. If the license plate numbers are consistent, the vehicle can be further identified to determine whether it is a cloned license plate vehicle.

[0070] In the above-mentioned embodiment of the present application, the target object is determined based on the characteristic information of the mobile object, including: obtaining the time information and spatial information of the mobile object; determining a first object set based on the time information, spatial information and characteristic information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; determining the target object based on the first granularity feature and / or the second granularity feature of the first mobile object.

[0071] For example, let's take the example of identifying cloned vehicles in traffic monitoring. Since cloned vehicles forge and use the license plates of legitimate vehicles, there will often be conflicts in the time and space relationship between legitimate vehicles and cloned vehicles. For example, vehicles with the same license plate number appear at different locations at the same time; vehicles with the same license plate number appear at different locations within a period of time, and the vehicle cannot appear at different locations within the time period.

[0072] In an optional embodiment, the checkpoint device captures the first image at a fixed time, and therefore, the time information of all moving objects contained in the first image can be determined based on the capture time of the first image; the position of each checkpoint device is fixed, and the actual positions of moving objects at different positions in the image on the traffic road are also different, and therefore, the spatial information of all moving objects contained in the first image can be determined based on the latitude and longitude positions of the checkpoint device and the positions of all moving objects identified and calibrated by the first image.

[0073] The first object set in the above steps may be a suspect object set, and all suspect objects in the same set have the same identification information. For example, still taking the example of duplicate license plate identification in traffic monitoring, the first object set may be duplicate license plate pairs.

[0074] In an optional embodiment, after identifying the identification information, the first granularity feature, the second granularity feature and other information, the server can first determine the first object set contained in the first image by comparing and detecting the spatiotemporal information and feature information of the moving objects. Furthermore, it is necessary to detect the first granularity feature and the second granularity feature of all the moving objects in the first object set to determine the cloned cars in the cloned pair.

[0075] In the above-mentioned embodiment of the present application, a first object set is determined based on the time information, spatial information and feature information of the mobile object, including: determining a second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; judging whether the time information and spatial information of the multiple second mobile objects meet the preset conditions, and matching the first granularity features and the second granularity features of the multiple second mobile objects; if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the first granularity features and the second granularity features of the multiple second mobile objects fail to match, then determining that the second object set is the first object set.

[0076] The second object set in the above steps may refer to a set of mobile objects using the same identification information. Each mobile object in the set may represent a mobile object photographed by the camera device at different times and / or different locations. Therefore, all mobile objects contained in the same second object set may be the same legal mobile object, or may be legal mobile objects and illegal mobile objects.

[0077] The preset condition in the above steps may be a condition for determining that there is a contradiction in the spatiotemporal relationship of all mobile objects in the same second object set, for example, the spatial distance between the mobile objects is greater than the maximum distance that the mobile objects can travel within the time interval, but is not limited to this.

[0078] For example, still taking the identification of duplicate license plate vehicles in traffic monitoring as an example, in order to achieve duplicate license plate vehicle identification, we can first determine the set of vehicles using the same license plate number, and further determine whether the spatiotemporal relationship of all vehicles is contradictory. If the spatiotemporal relationship of the vehicles is contradictory, it is determined that there are duplicate license plate vehicles in the vehicle set; if the spatiotemporal relationship of the vehicles is not contradictory, it can be further determined whether the vehicle characteristics and vehicle attributes are the same. If they are the same, it indicates that all vehicles are the same vehicle and not duplicate license plate vehicles; if they are not the same, it is determined that there are duplicate license plate vehicles in the vehicle set, thereby achieving the purpose of generating duplicate license plate pairs.

[0079] In the above-mentioned embodiment of the present application, determining the target object based on the first granularity feature and / or the second granularity feature of the first mobile object includes at least one of the following: comparing the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determining the target object based on the comparison result; matching the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determining the target object based on the matching result; matching the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determining the target object based on the matching result; processing the historical driving trajectory of the first mobile object to determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0080] For example, using the identification of cloned vehicles in traffic monitoring as an example, the registration information mentioned above can be the structured information registered when the vehicle was registered, including, but not limited to, the color, model, brand, and driver's address. The registration image can be a photo taken when the vehicle was registered. The historical image can be images captured by the camera during the vehicle's historical driving process.

[0081] It should be noted that the number of historical images obtained and the time difference with the current time can be determined according to actual recognition needs, and this application does not make specific restrictions on this.

[0082] The four comparison methods provided in the above embodiment of the present application can be used individually or in combination. In an optional embodiment, the four comparison methods are used in combination as an example for illustration. For example, the identification of duplicate license plates in traffic monitoring is still used as an example. Figure 5 As shown, queryA represents the image of vehicle A in the deck pair, and queryB represents the image of vehicle B in the deck pair.

[0083] First, the identified vehicle attributes can be compared with the registration information corresponding to the license plate number. That is, the registration structured information of the legal vehicle can be compared with the vehicle structured information of the cloned license plate pair. If the comparison result of a certain information is inconsistent, there is no need to perform other comparison methods, and the vehicle can be directly determined to be a cloned license plate vehicle; if all information is consistent, other comparison methods can be continued.

[0084] Secondly, the vehicle features of the first vehicle can be matched with the registration image corresponding to the license plate number, that is, the vehicle features identified in the registration image of the legitimate vehicle are compared with the vehicle features of the cloned license plate pair respectively. If the match fails, there is no need to perform other comparison methods and the vehicle can be directly determined to be a cloned license plate vehicle; if the match is successful, other comparison methods can be continued.

[0085] Again, multiple historical images of vehicles with the same license plate number can be extracted, and the vehicle features of the first vehicle can be matched with the multiple historical images corresponding to the license plate number. That is, the identified vehicle features can be compared with the vehicle features of the cloned license plate pair respectively. If the match fails, there is no need to perform other comparison methods, and the vehicle can be directly determined to be a cloned license plate vehicle; if the match is successful, other comparison methods can be continued.

[0086] Finally, the historical driving trajectory of the first vehicle can be processed, that is, the historical driving trajectory in the historical process record can be analyzed and integrated with the address information of the vehicle registration to identify the cloned license plate vehicle. Among them, if the driving trajectory of the cloned license plate pair fails to match the historical driving trajectory, then the vehicle is determined to be a cloned license plate vehicle. Alternatively, if the driving trajectory of the cloned license plate pair matches the historical driving trajectory but fails to match the address information of the vehicle registration, then the vehicle can also be determined to be a cloned license plate vehicle.

[0087] Through the above steps, machine learning technology and big data technology can be used to re-identify suspicious vehicles and ultimately determine whether the vehicles have cloned license plates, thereby reducing the workload of inspectors and improving identification efficiency.

[0088] In the above embodiment of the present application, the second granularity feature of the first moving object is matched with the registration image corresponding to the identification information, and the target object is determined based on the matching result, including: processing the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; when the feature distance is less than or equal to a preset value, determining that the matching result is a successful match; when the feature distance is greater than the preset value, determining that the matching result is a failed match.

[0089] The more similar two features are, the smaller the feature distance between the two features is. Therefore, in order to achieve feature matching, a preset value can be set in advance to determine whether the two features match.

[0090] For example, still taking the identification of cloned vehicles in traffic monitoring as an example, in an optional embodiment, the registration image can be processed using a vehicle feature extraction model based on the attention mechanism to extract the vehicle features of the registered vehicle, further obtain the feature distance between the vehicle features of the cloned license pair and the vehicle features of the registered vehicle, and further identify the cloned vehicle based on the feature distance. If the feature distance of a certain vehicle feature is greater than a preset value, it indicates that the vehicle features of the registered vehicle and the vehicle features of the cloned license pair fail to match; if the feature distances of all features are less than or equal to the preset value, it indicates that the vehicle features of the registered vehicle and the vehicle features of the cloned license pair successfully match.

[0091] In the above embodiment of the present application, the second granularity feature of the first moving object is matched with multiple historical images corresponding to the identification information, and the target object is determined based on the matching result, including: using a third network model to process the historical image to obtain the second granularity feature of the historical moving object contained in the historical image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in the multiple historical images to obtain multiple feature distances; sorting the multiple feature distances, and using the nearest neighbor algorithm to process the sorted feature distances to obtain a matching result.

[0092] For example, still taking the identification of cloned license plates in traffic monitoring as an example, in an optional embodiment, the historical images can be processed using a vehicle feature extraction model based on the attention mechanism to extract the vehicle features of the historical vehicles, and further obtain the feature distance between the vehicle features of the cloned license plate pair and the vehicle features of all historical vehicles, and sort them according to the feature distance, and use the nearest neighbor algorithm to identify cloned license plates. For example, after sorting the feature distances, it can be determined whether the vehicle features of the cloned license plate pair match the vehicle features of each historical vehicle based on the comparison results of each feature distance with the preset value. The nearest neighbor algorithm can be further used to determine whether the vehicle features of the cloned license plate pair match the vehicle features of all historical vehicles. If the match fails, the vehicle is determined to be a cloned license plate vehicle; if the match is successful, other comparison methods can be continued.

[0093] In the above embodiment of the present application, after determining the target object based on the characteristic information of the moving object, the characteristic information of the target object is stored in a database. The method also includes the following steps: obtaining a second image of the moving object to be identified; processing the second image to obtain the characteristic information of the moving object to be identified; and when the characteristic information of the moving object to be identified successfully matches the characteristic information of any target object stored in the database, determining that the moving object to be identified is the target object.

[0094] The second image in the above step can be an image containing the mobile object to be identified, captured by a checkpoint device installed on a traffic road, or can be a surveillance video containing the mobile object to be identified, captured by the checkpoint device. The database in the above step can be a blacklist of illegal mobile objects, storing identification information used by illegal mobile objects, as well as first and second granular features of the illegal mobile objects.

[0095] For example, let’s take the example of identifying duplicate license plate vehicles in traffic monitoring. Figure 6 As shown, after a cloned car is identified, the characteristic information of the cloned car can be added to a blacklist to facilitate subsequent rapid identification of the cloned car.

[0096] In an optional embodiment, as Figure 6As shown, the camera device captures a second image containing the vehicle to be identified and uploads it to the server. The server can process the second image through the above-mentioned first network model, second network model and third network model to obtain the vehicle information of the vehicle to be identified, and further compare and analyze the vehicle information of the vehicle to be identified with the vehicle information stored in the blacklist in multiple dimensions to verify whether the vehicle to be identified is a cloned license plate vehicle, specifically including attribute, license plate number and feature verification.

[0097] In the above embodiment of the present application, after determining that the mobile object to be identified is a target object, the method further includes the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; based on the driving trajectory and / or driving characteristics in the future period of time, generating alarm information, and capturing the mobile object to be identified.

[0098] The future period of time in the above steps can be determined based on actual trajectory prediction needs, and this application does not impose specific limitations on this. The driving area can be a frequently seen location of the target object, the parking area can be a frequently stopped location of the target object, and the driving time can be a frequently seen time period of the target object. The alarm information can also include the driving trajectory and / or driving characteristics for the future period of time.

[0099] For example, using the example of identifying cloned vehicles in traffic monitoring, in an optional embodiment, big data technology can be used to analyze the historical driving trajectories of cloned vehicles to identify their driving characteristics, specifically, but not limited to, their frequent appearance locations, frequent stops, and frequent appearance times. The discovered driving characteristics are then reported to inspectors via an alarm, facilitating their inspection of the cloned vehicles.

[0100] In another optional embodiment, as Figure 6 As shown, big data technology can be used to predict the next driving trajectory of the cloned vehicle based on its historical driving trajectory, and report it to the inspectors through an alarm, making it easier for the inspectors to check the cloned vehicle.

[0101] Through the above steps, big data technology is used to mine and analyze vehicle behavior, and the driving trajectory is immediately predicted when a cloned license plate vehicle is discovered, so that the inspection personnel can cooperate to complete the precise control behavior.

[0102] It should be noted that the historical driving trajectory in the above steps can be generated based on the acquisition time and location of the historical images containing the moving object.

[0103] In the above embodiment of the present application, after the mobile object to be identified is successfully found, the feature information of the mobile object to be identified stored in the database is deleted.

[0104] For example, still taking the identification of cloned vehicles in traffic monitoring as an example, in an optional embodiment, Figure 6 As shown, after the inspector successfully checks the vehicle with a fake license plate, the vehicle information of the vehicle with the fake license plate can be deleted from the blacklist to release the storage space of the blacklist.

[0105] The following combination Figure 7 , still taking the identification of cloned license plate vehicles in traffic monitoring as an example, a preferred embodiment of the present application is described in detail.

[0106] like Figure 7 As shown in FIG, the target object recognition method mainly includes the following four steps: deck pair generation, deck identity determination, trajectory mining / prediction, and real-time control.

[0107] The license plate pair generation process can include the following five steps: data acquisition and processing, vehicle target detection, license plate recognition, vehicle detail feature extraction, and spatiotemporal collision and visual feature comparison. Data acquisition and processing: The server obtains the first image of the vehicle captured in real time by the checkpoint device and the license plate number recognized by the checkpoint device; vehicle target detection: A deep convolutional neural network is used to identify the attributes and calibrate the positions of all vehicles in the first image; license plate recognition: A large-scale cloud computing platform is used to perform high-precision secondary recognition of the license plate and compare it with the license plate recognized by the device. If a match is found, the following steps are performed; Vehicle detail feature extraction: The first image is input into a deep convolutional neural network to obtain vehicle image features at different scales, thereby obtaining high-dimensional features of the vehicle. The vehicle's coarse-grained features are used for supervision, and the high-dimensional features are combined with the coarse-grained features and converted into feature channels with different weights through the attention module. A recurrent neural network is used to learn contextual association information of different feature channels, and vehicle features are obtained by optimizing the loss of coarse-grained vehicle categories and fine-grained categories. Spatiotemporal collision and visual feature comparison: Comparative detection is performed based on the spatiotemporal information, vehicle characteristics, and vehicle attributes of the same license plate number, and the comparison outputs the license plate pairs.

[0108] The process of identifying duplicate license plates can include the following four steps: structured information comparison, single-point image feature comparison, multi-point image feature comparison, and historical vehicle behavior analysis and comparison. Structured information comparison involves comparing the structured registration information of the real vehicle with the structured attribute information of the duplicate license plate pair to identify duplicate license plates. Single-point image feature comparison involves comparing the high-dimensional features of the real vehicle registration image with the high-dimensional features of the duplicate license plate pair image, identifying duplicate license plates based on feature distance. Multi-point image feature comparison involves extracting high-dimensional features from historical vehicle records, comparing them with the high-dimensional features of the duplicate license plate pair image, sorting them based on feature distance, and using the nearest neighbor algorithm to identify duplicate license plates. Historical vehicle behavior analysis and comparison involves analyzing the vehicle driving trajectories in historical vehicle records and integrating them with the vehicle registration address information to identify duplicate license plates.

[0109] The implementation process of trajectory mining / prediction is as follows: using big data technology to analyze the behavioral characteristics of cloned vehicles, and to discover high-frequency appearance points, destinations, and high-frequency appearance time periods; using big data technology to predict the next driving trajectory of the cloned vehicles based on historical driving trajectories.

[0110] The implementation process of real-time control is as follows: the vehicle information of the cloned license plate vehicle is added to the blacklist after analysis and confirmation; the vehicle information of the vehicle captured by the checkpoint equipment is compared with the vehicle information on the blacklist in multiple dimensions to verify whether the captured vehicle is a cloned license plate vehicle; big data technology is used to predict the next driving trajectory of the cloned license plate vehicle based on the driving direction of historical vehicle passing records; an alarm is issued, and the cargo of the cloned license plate vehicle is checked; after the vehicle is successfully seized, the vehicle information is removed from the blacklist.

[0111] Through the above scheme, a method for identifying and controlling cloned license plate vehicles in surveillance video has been proposed. Combined with multi-dimensional information processing, it can accurately detect, identify, analyze, and control cloned license plate vehicles in surveillance video, and cooperate with the police to effectively deal with cloned license plate vehicles, protecting the legitimate rights and interests of the real car owners and maintaining market order and social stability.

[0112] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0114] Example 2

[0115] According to an embodiment of the present application, a method for identifying a target object is also provided.

[0116] Figure 8 FIG. 1 is a flow chart of a second target object identification method according to an embodiment of the present application. Figure 8 As shown, the method includes the following steps:

[0117] Step S802, obtaining a second image of the moving object to be identified;

[0118] The moving objects in the above steps can be any movable object such as a vehicle or pedestrian in a traffic monitoring scene. In the embodiments of this application, a vehicle is used as an example for illustration. The second image can be an image containing the moving object to be identified, captured by a checkpoint device installed on a traffic road, or it can be a surveillance video containing the moving object to be identified, captured by the checkpoint device. The checkpoint device can capture images at fixed time intervals and can periodically upload the captured images to a server according to the capture time, which then performs mobile object identification.

[0119] Step S804: Process the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature;

[0120] The first granularity feature in the above steps may refer to the coarse-grained feature of the moving object. For example, for a vehicle, the feature may include at least one of the following: color, model and brand, but not limited to these. The second granularity feature may refer to the fine-grained feature of the moving object. For example, for a vehicle, the feature may include at least one of the following: annual inspection mark, decorations and windows, but not limited to these.

[0121] Step S806, matching the feature information of the mobile object to be identified with the feature information of at least one target object stored in the database;

[0122] The database in the above steps may be a blacklist of illegal mobile objects, and the database stores identification information used by the illegal mobile objects and first granularity features and second granularity features of the illegal mobile objects.

[0123] Step S808: If the feature information of the mobile object to be identified successfully matches the feature information of any target object stored in the database, the mobile object to be identified is determined to be the target object.

[0124] The target object in the above steps may be a moving object that needs to be identified, for example, an illegal vehicle that needs to be monitored in a traffic monitoring scene, but is not limited thereto.

[0125] Based on the solution provided by the above-mentioned embodiment of the present application, after obtaining a second image of the mobile object to be identified, the second image can be processed to obtain feature information such as the first granularity feature and the second granularity feature of the mobile object to be identified, and the feature information of the mobile object can be matched with the feature information of at least one target object stored in the database. Finally, the target object is determined based on the matching result, thereby achieving the purpose of mobile object identification. It is easy to notice that because the feature information contains the first granularity feature and the second granularity feature, the mobile object can be identified in multiple dimensions, thereby achieving the effect of improving the accuracy of mobile object identification and enhancing the efficiency of mobile object identification, thereby solving the technical problem of low accuracy of illegal vehicle identification in the target object identification method in the prior art.

[0126] In the above embodiment of the present application, the second image is processed to obtain feature information of the moving object to be identified, including: using the first network model to process the second image to obtain the first granularity feature of the moving object to be identified; using the second network model to process the second image to obtain the second granularity feature of the moving object to be identified, wherein the second network model is trained using the first granularity feature.

[0127] The first network model in the above steps can be a trained deep convolutional neural network. This application does not limit the specific structure of the deep convolutional neural network, which can be determined according to actual use requirements and recognition accuracy requirements. The second network model can be a second granularity feature extraction model based on the attention mechanism, including: a first sub-network model, a second sub-network model and a third sub-network model, wherein the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network.

[0128] In the above embodiment of the present application, the second image is processed using the second network model to obtain the second granularity feature of the moving object to be identified, including: using the first sub-network model to process the second image to obtain the first feature of the moving object to be identified; using the second sub-network model to process the second image to obtain the second feature of the moving object to be identified; combining the first feature and the second feature to obtain a combined feature; using the attention module to process the combined feature to obtain multiple feature channels, wherein different feature channels have different weight values; using the third sub-network model to process the multiple feature channels to obtain the second granularity feature of the moving object to be identified.

[0129] In the above embodiment of the present application, before using the second network model to process the second image to obtain the second granularity feature of the moving object to be identified, the method also includes the following steps: using the third network model to process the second image to obtain the identification information of the moving object to be identified; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, then using the third network model to process the second image to obtain the second granularity feature of the moving object to be identified.

[0130] The third network model in the above step can be a trained deep convolutional neural network. This application does not limit the specific structure of the deep convolutional neural network and can be determined based on actual usage requirements and recognition accuracy requirements. The identification information can be a vehicle license plate number, a pedestrian ID, etc., but is not limited to this.

[0131] In the above embodiment of the present application, after determining that the mobile object to be identified is a target object, the method further includes the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; based on the driving trajectory and / or driving characteristics in the future period of time, generating alarm information, and capturing the mobile object to be identified.

[0132] The future period of time in the above steps can be determined based on actual trajectory prediction needs, and this application does not impose specific limitations on this. The driving area can be a frequently seen location of the target object, the parking area can be a frequently stopped location of the target object, and the driving time can be a frequently seen time period of the target object. The alarm information can also include the driving trajectory and / or driving characteristics for the future period of time.

[0133] In the above embodiment of the present application, the method also includes the following steps: acquiring multiple captured first images, wherein the first image contains at least one moving object; processing the first image to obtain characteristic information of the moving object, wherein the characteristic information includes: a first granularity feature and a second granularity feature; determining a target object based on the characteristic information of the moving object; and storing the characteristic information of the target object in a database.

[0134] The first image in the above step may be an image containing a moving object captured by a checkpoint device installed on a traffic road, or may be a surveillance video containing a moving object captured by the checkpoint device.

[0135] In the above-mentioned embodiment of the present application, the target object is determined based on the characteristic information of the mobile object, including: obtaining the time information and spatial information of the mobile object; determining a first object set based on the time information, spatial information and characteristic information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; determining the target object based on the first granularity feature and / or the second granularity feature of the first mobile object.

[0136] The first object set in the above steps may be a suspect object set, and all suspect objects in the same set have the same identification information. For example, still taking the example of duplicate license plate identification in traffic monitoring, the first object set may be duplicate license plate pairs.

[0137] In the above-mentioned embodiment of the present application, a first object set is determined based on the time information, spatial information and feature information of the mobile object, including: determining a second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; judging whether the time information and spatial information of the multiple second mobile objects meet the preset conditions, and matching the first granularity features and the second granularity features of the multiple second mobile objects; if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the first granularity features and the second granularity features of the multiple second mobile objects fail to match, then determining that the second object set is the first object set.

[0138] The second object set in the above steps may refer to a set of mobile objects using the same identification information. Each mobile object in the set may represent a mobile object photographed by the camera device at different times and / or different locations. Therefore, all mobile objects contained in the same second mobile object set may be the same legal mobile object, or may be legal mobile objects and illegal mobile objects.

[0139] The preset condition in the above steps may be a condition for determining that there is a contradiction in the spatiotemporal relationship of all mobile objects in the same second object set, for example, the spatial distance between the mobile objects is greater than the maximum distance that the mobile objects can travel within the time interval, but is not limited to this.

[0140] In the above-mentioned embodiment of the present application, the target object is determined based on the first granularity feature and the second granularity feature of the first mobile object, including at least one of the following: comparing the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determining the target object based on the comparison result; matching the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determining the target object based on the matching result; matching the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determining the target object based on the matching result; processing the historical driving trajectory of the first mobile object to determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0141] For example, using the identification of cloned vehicles in traffic monitoring as an example, the registration information mentioned above can be the structured information registered when the vehicle was registered, including, but not limited to, the color, model, brand, and driver's address. The registration image can be a photo taken when the vehicle was registered. The historical image can be images captured by the camera during the vehicle's historical driving process.

[0142] In the above embodiment of the present application, the second granularity feature of the first moving object is matched with the registration image corresponding to the identification information, and the target object is determined based on the matching result, including: processing the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; when the feature distance is less than or equal to a preset value, determining that the matching result is a successful match; when the feature distance is greater than the preset value, determining that the matching result is a failed match.

[0143] The more similar two features are, the smaller the feature distance between the two features is. Therefore, in order to achieve feature matching, a preset value can be set in advance to determine whether the two features match.

[0144] In the above embodiment of the present application, the second granularity feature of the first moving object is matched with multiple historical images corresponding to the identification information, and the target object is determined based on the matching result, including: using a third network model to process the historical image to obtain the second granularity feature of the historical moving object contained in the historical image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in the multiple historical images to obtain multiple feature distances; sorting the multiple feature distances, and using the nearest neighbor algorithm to process the sorted feature distances to obtain a matching result.

[0145] In the above embodiment of the present application, after the mobile object to be identified is successfully found, the feature information of the mobile object to be identified stored in the database is deleted.

[0146] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0147] Example 3

[0148] According to an embodiment of the present application, a target object identification device for implementing the above target object identification method is also provided. Figure 9 As shown, the apparatus 900 includes: a first acquisition module 902 , a first processing module 904 and a first determination module 906 .

[0149] Among them, the first acquisition module 902 is used to acquire multiple collected first images, wherein the first image contains at least one moving object; the first processing module 904 is used to process the first image to obtain characteristic information of the moving object, wherein the characteristic information includes: a first granularity feature and a second granularity feature; the first determination module 906 is used to determine the target object based on the characteristic information of the moving object.

[0150] It should be noted that the first acquisition module 902, the first processing module 904, and the first determination module 906 correspond to steps S302 to S306 in Example 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0151] In the above embodiment of the present application, the first processing module includes: a first processing sub-module and a second processing sub-module.

[0152] Among them, the first processing submodule is used to process the first image using the first network model to obtain the first granularity feature of the moving object; the second processing submodule is used to process the first image using the second network model to obtain the second granularity feature of the moving object, wherein the second network model is trained using the first granularity feature.

[0153] In the above embodiment of the present application, the second processing submodule includes: a first processing unit, a second processing unit, a combining unit, a third processing unit and a fourth processing unit.

[0154] Among them, the first processing unit is used to process the first image using the first sub-network model to obtain the first feature of the moving object; the second processing unit is used to process the first image using the second sub-network model to obtain the second feature of the moving object; the combination unit is used to combine the first feature and the second feature to obtain a combined feature; the third processing unit is used to process the combined feature using the attention module to obtain multiple feature channels, wherein different feature channels have different weight values; the fourth processing unit is used to process the multiple feature channels using the third sub-network model to obtain the second granularity feature of the moving object.

[0155] In the above embodiment of the present application, the first processing module further includes: a third processing submodule, a first comparison submodule and a fourth processing submodule.

[0156] Among them, the third processing submodule is used to process the first image using the third network model to obtain the identification information of the moving object; the first comparison submodule is used to compare the processed identification information with the received identification information; the fourth processing submodule is used to process the first image using the third network model to obtain the second granularity feature of the moving object if the processed identification information is consistent with the received identification information.

[0157] In the above embodiment of the present application, the first determination module includes: an acquisition submodule, a first determination submodule and a second determination submodule.

[0158] Among them, the acquisition submodule is used to obtain the time information and spatial information of the mobile object; the first determination submodule is used to determine the first object set based on the time information, spatial information and feature information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; the second determination submodule is used to determine the target object based on the first granularity feature and / or the second granularity feature of the first mobile object.

[0159] In the above embodiment of the present application, the first determining submodule includes: a first determining unit, a judging unit and a second determining unit.

[0160] Among them, the first determination unit is used to determine the second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; the judgment unit is used to judge whether the time information and spatial information of the multiple second mobile objects meet the preset conditions, and match the first granularity features and the second granularity features of the multiple second mobile objects; the second determination unit is used to determine that the second object set is the first object set if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the first granularity features and the second granularity features of the multiple second mobile objects fail to match.

[0161] In the above embodiment of the present application, the second determining submodule includes at least one of the following: a comparing unit, a first matching unit, a second matching unit, and a third determining unit.

[0162] Among them, the comparison unit is used to compare the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determine the target object based on the comparison result; the first matching unit is used to match the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determine the target object based on the matching result; the second matching unit is used to match the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determine the target object based on the matching result; the third determination unit is used to process the historical driving trajectory of the first mobile object and determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0163] In the above embodiment of the present application, the first matching unit includes: a first processing subunit, a first acquiring subunit, a first determining subunit, and a second determining subunit.

[0164] Among them, the first processing subunit is used to process the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; the first acquisition subunit is used to obtain the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; the first determination subunit is used to determine that the matching result is a successful match when the feature distance is less than or equal to a preset value; the second determination subunit is used to determine that the matching result is a failed match when the feature distance is greater than a preset value.

[0165] In the above embodiment of the present application, the second matching unit includes: a second processing subunit, a second acquiring subunit and a third processing subunit.

[0166] Among them, the second processing subunit is used to process the historical image using the third network model to obtain the second granularity feature of the historical moving object contained in the historical image; the second acquisition subunit is used to obtain the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in multiple historical images to obtain multiple feature distances; the third processing subunit is used to sort the multiple feature distances, and use the nearest neighbor algorithm to process the sorted feature distances to obtain matching results.

[0167] In the above embodiment of the present application, the device further includes: a storage module and a second determination module.

[0168] Among them, the storage module is used to store the characteristic information of the target object in the database; the first acquisition module is also used to obtain a second image of the moving object to be identified; the first processing module is also used to process the second image to obtain the characteristic information of the moving object to be identified; the second determination module is used to determine that the moving object to be identified is the target object when the characteristic information of the moving object to be identified successfully matches the characteristic information of any target object stored in the database.

[0169] In the above embodiment of the present application, the device further includes: a second acquisition module, a second processing module and a generation module.

[0170] Among them, the second acquisition module is used to obtain the historical driving trajectory of the mobile object to be identified; the second processing module is used to process the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; the generation module is used to generate alarm information based on the driving trajectory and / or driving characteristics in the future period of time, and to capture the mobile object to be identified.

[0171] In the above embodiment of the present application, the device further includes: a deletion module.

[0172] The deletion module is used to delete the feature information of the mobile object to be identified stored in the database after the mobile object to be identified is successfully found.

[0173] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0174] Example 4

[0175] According to an embodiment of the present application, a target object identification device for implementing the above target object identification method is also provided. Figure 10As shown, the apparatus 1000 includes: a first acquisition module 1002 , a first processing module 1004 , a matching module 1006 and a determination module 1008 .

[0176] Among them, the first acquisition module 1002 is used to acquire a second image of the moving object to be identified; the first processing module 1004 is used to process the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature; the matching module 1006 is used to match the feature information of the moving object to be identified with the feature information of at least one target object stored in the database; the determination module 1008 is used to determine that the moving object to be identified is the target object if the feature information of the moving object to be identified successfully matches the feature information of any target object stored in the database.

[0177] It should be noted that the first acquisition module 1002, first processing module 1004, matching module 1006, and determination module 1008 described above correspond to steps S802 to S808 in Example 2. The examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to those disclosed in Example 1. It should be noted that the above modules, as part of the apparatus, can be run in the computer terminal 10 provided in Example 1.

[0178] In the above embodiment of the present application, the first processing module includes: a first processing sub-module and a second processing sub-module.

[0179] Among them, the first processing submodule is used to process the second image using the first network model to obtain the first granularity feature of the moving object to be identified; the second processing submodule is used to process the second image using the second network model to obtain the second granularity feature of the moving object to be identified, wherein the second network model is trained using the first granularity feature.

[0180] In the above embodiment of the present application, the second processing submodule includes: a first processing unit, a second processing unit, a combining unit, a third processing unit and a fourth processing unit.

[0181] Among them, the first processing unit is used to process the second image using the first sub-network model to obtain the first feature of the moving object to be identified; the second processing unit is used to process the second image using the second sub-network model to obtain the second feature of the moving object to be identified; the combination unit is used to combine the first feature and the second feature to obtain a combined feature; the third processing unit is used to process the combined feature using the attention module to obtain multiple feature channels, wherein different feature channels have different weight values; the fourth processing unit is used to process the multiple feature channels using the third sub-network model to obtain the second granularity feature of the moving object to be identified.

[0182] In the above embodiment of the present application, the first processing module further includes: a third processing submodule, a first comparison submodule and a fourth processing submodule.

[0183] Among them, the third processing submodule is used to process the second image using the third network model to obtain the identification information of the moving object to be identified; the first comparison submodule is used to compare the processed identification information with the received identification information; the fourth processing submodule is used to process the first image using the third network model if the processed identification information is consistent with the received identification information to obtain the second granularity feature of the moving object to be identified.

[0184] In the above embodiment of the present application, the device further includes: a second acquisition module, a second processing module and a generation module.

[0185] Among them, the second acquisition module is used to obtain the historical driving trajectory of the mobile object to be identified; the second processing module is used to process the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; the generation module is used to generate alarm information based on the driving trajectory and / or driving characteristics in the future period of time, and to capture the mobile object to be identified.

[0186] In the above embodiment of the present application, the device further includes: a second determination module and a storage module.

[0187] Among them, the first acquisition module is also used to acquire multiple collected first images, wherein the first image contains at least one moving object; the first processing module is also used to process the first image to obtain characteristic information of the moving object, wherein the characteristic information includes: a first granularity feature and a second granularity feature; the second determination module is used to determine the target object based on the characteristic information of the moving object; the storage module is also used to store the characteristic information of the target object in a database.

[0188] In the above embodiment of the present application, the second determination module includes: an acquisition submodule, a first determination submodule and a second determination submodule.

[0189] Among them, the acquisition submodule is used to obtain the time information and spatial information of the mobile object; the first determination submodule is used to determine the first object set based on the time information, spatial information and feature information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; the second determination submodule is used to determine the target object based on the first granularity feature and the second granularity feature of the first mobile object.

[0190] In the above embodiment of the present application, the first determining submodule includes: a first determining unit, a judging unit and a second determining unit.

[0191] Among them, the first determination unit is used to determine the second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; the judgment unit is used to judge whether the time information and spatial information of the multiple second mobile objects meet the preset conditions, and match the first granularity features and the second granularity features of the multiple second mobile objects; the second determination unit is used to determine that the second object set is the first object set if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the first granularity features and the second granularity features of the multiple second mobile objects fail to match.

[0192] In the above embodiment of the present application, the second determining submodule includes at least one of the following: a comparing unit, a first matching unit, a second matching unit, and a third determining unit.

[0193] Among them, the comparison unit is used to compare the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determine the target object based on the comparison result; the first matching unit is used to match the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determine the target object based on the matching result; the second matching unit is used to match the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determine the target object based on the matching result; the third determination unit is used to process the historical driving trajectory of the first mobile object and determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0194] In the above embodiment of the present application, the first matching unit includes: a first processing subunit, a first acquiring subunit, a first determining subunit, and a second determining subunit.

[0195] Among them, the first processing subunit is used to process the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; the first acquisition subunit is used to obtain the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; the first determination subunit is used to determine that the matching result is a successful match when the feature distance is less than or equal to a preset value; the second determination subunit is used to determine that the matching result is a failed match when the feature distance is greater than a preset value.

[0196] In the above embodiment of the present application, the second matching unit includes: a second processing subunit, a second acquiring subunit and a third processing subunit.

[0197] Among them, the second processing subunit is used to process the historical image using the third network model to obtain the second granularity feature of the historical moving object contained in the historical image; the second acquisition subunit is used to obtain the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in multiple historical images to obtain multiple feature distances; the third processing subunit is used to sort the multiple feature distances, and use the nearest neighbor algorithm to process the sorted feature distances to obtain matching results.

[0198] In the above embodiment of the present application, the device further includes: a deletion module.

[0199] The deletion module is used to delete the feature information of the mobile object to be identified stored in the database after the mobile object to be identified is successfully found.

[0200] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0201] Example 5

[0202] According to an embodiment of the present application, a target object recognition system is further provided, including:

[0203] processor; and

[0204] The memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: obtaining multiple captured first images, wherein the first images contain at least one moving object; processing the first images to obtain feature information of the moving objects, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining a target object based on the feature information of the moving object.

[0205] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0206] Example 6

[0207] According to an embodiment of the present application, a method for identifying a target object is also provided.

[0208] Figure 11 FIG. 1 is a flow chart of a third target object identification method according to an embodiment of the present application. Figure 11 As shown, the method includes the following steps:

[0209] Step S1102: Acquire a plurality of captured first images, wherein each first image contains at least one moving object;

[0210] Step S1104: Processing the first image using the first network model to obtain a first granularity feature of the moving object;

[0211] Step S1106: Process the first image using a second network model to obtain a second granularity feature of the moving object, wherein the second network model includes: a first sub-network model, a second sub-network model, and a third sub-network model. The first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, and the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network.

[0212] Step S1108: Determine the target object based on the feature information of the moving object.

[0213] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0214] Example 7

[0215] According to an embodiment of the present application, a method for identifying a target object is also provided.

[0216] Figure 12 FIG. 1 is a flow chart of a fourth target object identification method according to an embodiment of the present application. Figure 12 As shown, the method includes the following steps:

[0217] Step S1202: Acquire multiple captured first images, wherein each first image contains at least one object to be identified;

[0218] The object to be identified in the above steps may be an object that needs to be identified, for example, a vehicle, a pedestrian, an animal, etc., but is not limited thereto. In the embodiment of the present application, mobile objects such as vehicles and pedestrians are used as examples for description.

[0219] Step S1204: Process the first image to obtain feature information of the object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature;

[0220] Step S1206: Determine the target object based on the feature information of the object to be identified.

[0221] It should be noted that the preferred implementation scheme involved in the above embodiments of this application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0222] Example 8

[0223] The embodiment of the present application can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0224] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0225] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the target object identification method: obtaining multiple captured first images, wherein the first image contains at least one moving object; processing the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining the target object based on the feature information of the moving object.

[0226] Optionally, Figure 13 This is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 13 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1102 and a memory 1104.

[0227] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the target object identification method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned target object identification method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0228] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain multiple captured first images, wherein the first image contains at least one moving object; process the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; and determine the target object based on the feature information of the moving object.

[0229] Optionally, the processor may also execute the program code of the following steps: processing the first image using the first network model to obtain the first granularity feature of the moving object; processing the first image using the second network model to obtain the second granularity feature of the moving object, wherein the second network model is trained using the first granularity feature.

[0230] Optionally, the processor may also execute the program code of the following steps: using the first sub-network model to process the first image to obtain the first feature of the moving object; using the second sub-network model to process the first image to obtain the second feature of the moving object; combining the first feature and the second feature to obtain a combined feature; using the attention module to process the combined feature to obtain multiple feature channels, wherein different feature channels have different weight values; using the third sub-network model to process the multiple feature channels to obtain the second granularity feature of the moving object.

[0231] Optionally, the processor may also execute the program code of the following steps: using a third network model to process the first image to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the third network model to process the first image to obtain a second granularity feature of the moving object.

[0232] Optionally, the above-mentioned processor can also execute the program code of the following steps: obtaining the time information and spatial information of the mobile object; determining the first object set based on the time information, spatial information and feature information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; determining the target object based on the first granularity feature and / or second granularity feature of the first mobile object.

[0233] Optionally, the processor may also execute the program code of the following steps: determining a second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; determining whether the time information and spatial information of the multiple second mobile objects meet preset conditions, and matching the first granularity features and the second granularity features of the multiple second mobile objects; if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the matching of the first granularity features and the second granularity features of the multiple second mobile objects fails, determining that the second object set is the first object set.

[0234] Optionally, the processor may also execute the program code of the following steps: comparing the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determining the target object based on the comparison result; and / or matching the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determining the target object based on the matching result; and / or matching the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determining the target object based on the matching result; and / or processing the historical driving trajectory of the first mobile object to determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0235] Optionally, the processor may also execute the program code of the following steps: processing the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; when the feature distance is less than or equal to a preset value, determining that the matching result is a successful match; when the feature distance is greater than the preset value, determining that the matching result is a failed match.

[0236] Optionally, the processor may also execute the program code of the following steps: using a third network model to process the historical image to obtain the second granularity feature of the historical moving object contained in the historical image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in multiple historical images to obtain multiple feature distances; sorting the multiple feature distances, and using the nearest neighbor algorithm to process the sorted feature distances to obtain matching results.

[0237] Optionally, the processor may also execute the program code of the following steps: obtaining a second image of the moving object to be identified; processing the second image to obtain characteristic information of the moving object to be identified; and determining that the moving object to be identified is the target object when the characteristic information of the moving object to be identified successfully matches the characteristic information of any target object stored in the database.

[0238] Optionally, the processor may also execute the program code of the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; generating alarm information based on the driving trajectory and / or driving characteristics in the future period of time, and capturing the mobile object to be identified.

[0239] Optionally, the processor may further execute a program code of the following steps: after the mobile object to be identified is successfully found, deleting the feature information of the mobile object to be identified stored in the database.

[0240] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a second image of the moving object to be identified; process the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature; match the feature information of the moving object to be identified with feature information of at least one target object stored in a database; if the feature information of the moving object to be identified successfully matches the feature information of any target object stored in the database, then determine that the moving object to be identified is the target object.

[0241] Optionally, the processor may also execute the program code of the following steps: processing the second image using the first network model to obtain the first granularity feature of the moving object to be identified; processing the second image using the second network model to obtain the second granularity feature of the moving object to be identified, wherein the second network model is trained using the first granularity feature.

[0242] Optionally, the processor may also execute the program code of the following steps: using the first sub-network model to process the second image to obtain the first feature of the moving object to be identified; using the second sub-network model to process the second image to obtain the second feature of the moving object to be identified; combining the first feature and the second feature to obtain a combined feature; using the attention module to process the combined feature to obtain multiple feature channels, wherein different feature channels have different weight values; using the third sub-network model to process the multiple feature channels to obtain the second granularity feature of the moving object to be identified.

[0243] Optionally, the processor may also execute the program code of the following steps: using the third network model to process the second image to obtain identification information of the mobile object to be identified; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the third network model to process the second image to obtain the second granularity feature of the mobile object to be identified.

[0244] Optionally, the processor may also execute the program code of the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; generating alarm information based on the driving trajectory and / or driving characteristics in the future period of time, and capturing the mobile object to be identified.

[0245] Optionally, the processor may also execute the program code of the following steps: acquiring multiple captured first images, wherein the first images contain at least one moving object; processing the first images to obtain characteristic information of the moving object, wherein the characteristic information includes: a first granularity feature and a second granularity feature; determining a target object based on the characteristic information of the moving object; and storing the characteristic information of the target object in a database.

[0246] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain multiple first images collected, wherein the first image contains at least one moving object; use the first network model to process the first image to obtain the first granularity feature of the moving object; use the second network model to process the first image to obtain the second granularity feature of the moving object, wherein the second network model includes: a first sub-network model, a second sub-network model and a third sub-network model, the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network; based on the characteristic information of the moving object, determine the target object.

[0247] The embodiments of the present application provide a solution for identifying moving objects. By processing a captured first image, characteristic information such as a first granularity feature and a second granularity feature of the moving object can be obtained, thereby identifying the moving object in multiple dimensions. This improves the accuracy and efficiency of mobile object identification, thereby resolving the technical problem of low accuracy in identifying illegal vehicles in existing target object identification methods.

[0248] It can be understood by those skilled in the art that Figure 13 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 13 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Figure 13 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 13 Different configurations shown.

[0249] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0250] Example 9

[0251] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the target object recognition method provided in the above embodiment.

[0252] Optionally, in this embodiment, the above-mentioned storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0253] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: acquiring multiple captured first images, wherein the first images contain at least one moving object; processing the first images to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; and determining a target object based on the feature information of the moving object.

[0254] Optionally, the storage medium is also configured to store program code for executing the following steps: processing the first image using the first network model to obtain the first granularity feature of the moving object; processing the first image using the second network model to obtain the second granularity feature of the moving object, wherein the second network model is trained using the first granularity feature.

[0255] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the first image using the first sub-network model to obtain a first feature of the moving object; processing the first image using the second sub-network model to obtain a second feature of the moving object; combining the first feature and the second feature to obtain a combined feature; processing the combined feature using the attention module to obtain multiple feature channels, wherein different feature channels have different weight values; and processing the multiple feature channels using the third sub-network model to obtain a second granularity feature of the moving object.

[0256] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the first image using a third network model to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, processing the first image using the third network model to obtain a second granularity feature of the moving object.

[0257] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: obtaining the time information and spatial information of the mobile object; determining a first object set based on the time information, spatial information and feature information of the mobile object, wherein the first object set includes: multiple first mobile objects corresponding to the same identification information; determining the target object based on the first granularity feature and / or second granularity feature of the first mobile object.

[0258] Optionally, the storage medium is also configured to store program code for executing the following steps: determining a second object set based on the identification information of the mobile object, wherein the second object set includes: multiple second mobile objects corresponding to the same identification information; judging whether the time information and spatial information of the multiple second mobile objects meet preset conditions, and matching the first granularity features and the second granularity features of the multiple second mobile objects; if the time information and spatial information of the multiple second mobile objects do not meet the preset conditions, or the first granularity features and the second granularity features of the multiple second mobile objects fail to match, then determining that the second object set is the first object set.

[0259] Optionally, the storage medium is also configured to store program codes for executing the following steps: comparing the first granularity feature of the first mobile object with the registration information corresponding to the identification information, and determining the target object based on the comparison result; and / or matching the second granularity feature of the first mobile object with the registration image corresponding to the identification information, and determining the target object based on the matching result; and / or matching the second granularity feature of the first mobile object with multiple historical images corresponding to the identification information, and determining the target object based on the matching result; and / or processing the historical driving trajectory of the first mobile object to determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on the first granularity feature and the second granularity feature of the first mobile object.

[0260] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the registration image using the second network model to obtain the second granularity feature of the registration object contained in the registration image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the registration object to obtain the feature distance; when the feature distance is less than or equal to a preset value, determining that the matching result is a successful match; when the feature distance is greater than the preset value, determining that the matching result is a failed match.

[0261] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the historical image using a third network model to obtain the second granularity feature of the historical moving object contained in the historical image; obtaining the distance between the second granularity feature of the first moving object and the second granularity feature of the historical moving object in multiple historical images to obtain multiple feature distances; sorting the multiple feature distances, and processing the sorted feature distances using a nearest neighbor algorithm to obtain a matching result.

[0262] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: obtaining a second image of the moving object to be identified; processing the second image to obtain feature information of the moving object to be identified; and determining that the moving object to be identified is the target object when the feature information of the moving object to be identified successfully matches the feature information of any target object stored in the database.

[0263] Optionally, the above-mentioned storage medium is also configured to store program codes for executing the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; based on the driving trajectory and / or driving characteristics in the future period of time, generating alarm information, and capturing the mobile object to be identified.

[0264] Optionally, the storage medium is further configured to store program codes for executing the following steps: after the mobile object to be identified is successfully found, deleting the feature information of the mobile object to be identified stored in the database.

[0265] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: obtaining a second image of the moving object to be identified; processing the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature; matching the feature information of the moving object to be identified with feature information of at least one target object stored in a database; if the feature information of the moving object to be identified successfully matches the feature information of any target object stored in the database, determining that the moving object to be identified is a target object.

[0266] Optionally, the storage medium is also configured to store program code for executing the following steps: processing the second image using the first network model to obtain the first granularity feature of the moving object to be identified; processing the second image using the second network model to obtain the second granularity feature of the moving object to be identified, wherein the second network model is trained using the first granularity feature.

[0267] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the second image using the first sub-network model to obtain a first feature of the moving object to be identified; processing the second image using the second sub-network model to obtain a second feature of the moving object to be identified; combining the first feature and the second feature to obtain a combined feature; processing the combined feature using the attention module to obtain multiple feature channels, wherein different feature channels have different weight values; and processing the multiple feature channels using the third sub-network model to obtain a second granularity feature of the moving object to be identified.

[0268] Optionally, the storage medium is also configured to store program codes for executing the following steps: processing the second image using a third network model to obtain identification information of the mobile object to be identified; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, processing the second image using the third network model to obtain a second granularity feature of the mobile object to be identified.

[0269] Optionally, the above-mentioned storage medium is also configured to store program codes for executing the following steps: obtaining the historical driving trajectory of the mobile object to be identified; processing the historical driving trajectory to obtain the driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: driving area, parking area and driving time; based on the driving trajectory and / or driving characteristics in the future period of time, generating alarm information, and capturing the mobile object to be identified.

[0270] Optionally, the storage medium is also configured to store program code for executing the following steps: acquiring multiple captured first images, wherein the first images contain at least one moving object; processing the first images to obtain characteristic information of the moving object, wherein the characteristic information includes: a first granularity feature and a second granularity feature; determining a target object based on the characteristic information of the moving object; and storing the characteristic information of the target object in a database.

[0271] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: acquiring multiple captured first images, wherein the first image contains at least one moving object; processing the first image using a first network model to obtain a first granularity feature of the moving object; processing the first image using a second network model to obtain a second granularity feature of the moving object, wherein the second network model includes: a first sub-network model, a second sub-network model and a third sub-network model, the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network; based on the feature information of the moving object, determine the target object.

[0272] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0273] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0274] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0275] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0276] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0277] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0278] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying a target object, comprising: Acquire a plurality of captured first images, wherein the first images contain at least one moving object; Processing the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature, wherein the second granularity feature is obtained by processing the first image using a second network model, and the second network model is trained using the first granularity feature; determining a target object based on the feature information of the moving object; The method also includes: using a third network model to process the first image to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the second network model to process the first image to obtain a second granularity feature of the moving object.

2. The method according to claim 1, wherein Processing the first image to obtain feature information of the moving object includes: The first image is processed using a first network model to obtain a first granularity feature of the moving object.

3. The method according to claim 2, wherein: Processing the first image using a second network model to obtain a second granularity feature of the moving object includes: Processing the first image using a first sub-network model to obtain a first feature of the moving object; Processing the first image using a second sub-network model to obtain a second feature of the moving object; combining the first feature and the second feature to obtain a combined feature; Processing the combined features using an attention module to obtain a plurality of feature channels, wherein different feature channels have different weight values; The plurality of feature channels are processed using a third sub-network model to obtain second granularity features of the moving object.

4. The method according to claim 1, wherein Determining a target object based on the feature information of the moving object includes: Acquiring time information and spatial information of the moving object; Determining a first object set based on the time information, spatial information, and feature information of the mobile object, wherein the first object set includes: a plurality of first mobile objects corresponding to the same identification information; The target object is determined based on the first granularity feature and / or the second granularity feature of the first moving object.

5. The method according to claim 4, wherein Determining a first object set based on the time information, spatial information, and feature information of the mobile objects includes: Determining a second object set based on the identification information of the mobile object, wherein the second object set includes: a plurality of second mobile objects corresponding to the same identification information; determining whether the time information and the spatial information of the plurality of second moving objects meet a preset condition, and matching the first granularity features with the second granularity features of the plurality of second moving objects; If the time information and the spatial information of the plurality of second moving objects do not meet the preset condition, or the first granularity features and the second granularity features of the plurality of second moving objects fail to match, the second object set is determined to be the first object set.

6. The method according to claim 4, wherein: Determining the target object based on the first granularity feature and / or the second granularity feature of the first moving object includes at least one of the following: comparing the first granularity feature of the first moving object with the registration information corresponding to the identification information, and determining the target object based on the comparison result; matching the second granularity feature of the first moving object with the registration image corresponding to the identification information, and determining the target object based on the matching result; matching the second granularity feature of the first moving object with a plurality of historical images corresponding to the identification information, and determining the target object based on the matching result; The historical driving trajectory of the first mobile object is processed to determine the target object, wherein the historical driving trajectory of the first mobile object is obtained based on a first granularity feature and a second granularity feature of the first mobile object.

7. The method according to claim 6, wherein: Matching the second granularity feature of the first moving object with the registered image corresponding to the identification information, and determining the target object based on the matching result, including: Processing the registration image using a second network model to obtain a second granularity feature of the registration object contained in the registration image; Acquire a distance between a second granularity feature of the first moving object and a second granularity feature of the registered object to obtain a characteristic distance; When the feature distance is less than or equal to a preset value, determining the matching result as a successful match; When the characteristic distance is greater than the preset value, the matching result is determined to be a matching failure.

8. The method according to claim 6, wherein: Matching the second granularity feature of the first moving object with a plurality of historical images corresponding to the identification information, and determining the target object based on the matching results, including: Processing the historical image using a second network model to obtain a second granularity feature of the historical moving object contained in the historical image; acquiring a distance between a second granularity feature of the first moving object and second granularity features of historical moving objects in the plurality of historical images to obtain a plurality of feature distances; The plurality of feature distances are sorted, and the sorted feature distances are processed using a nearest neighbor algorithm to obtain the matching result.

9. The method according to claim 1, wherein After determining the target object based on the characteristic information of the moving object, storing the characteristic information of the target object in a database, the method further includes: acquiring a second image of the moving object to be identified; processing the second image to obtain feature information of the moving object to be identified; In a case where the feature information of the mobile object to be identified successfully matches the feature information of any target object stored in the database, the mobile object to be identified is determined to be the target object.

10. The method according to claim 9, wherein: After determining that the moving object to be identified is a target object, the method further includes: Obtaining a historical driving trajectory of the mobile object to be identified; Processing the historical driving trajectory to obtain a driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: a driving area, a parking area, and a driving time; Based on the driving trajectory and / or the driving characteristics in the future period of time, an alarm message is generated, and the mobile object to be identified is captured.

11. The method according to claim 10, wherein: After the mobile object to be identified is successfully found, the feature information of the mobile object to be identified stored in the database is deleted.

12. A method for identifying a target object, comprising: acquiring a second image of the moving object to be identified; Processing the second image to obtain feature information of the moving object to be identified, wherein the feature information includes: a first granularity feature and a second granularity feature, wherein the second granularity feature is obtained by processing the second image using a second network model, and the second network model is trained using the first granularity feature; Matching the feature information of the mobile object to be identified with feature information of at least one target object stored in a database; If the feature information of the mobile object to be identified successfully matches the feature information of any target object stored in the database, determining that the mobile object to be identified is the target object; The method also includes: using a third network model to process the second image to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the second network model to process the second image to obtain a second granularity feature of the moving object.

13. The method according to claim 12, wherein: Processing the second image to obtain feature information of the moving object to be identified includes: The second image is processed using a first network model to obtain a first granularity feature of the moving object to be identified.

14. The method according to claim 13, wherein Processing the second image using a second network model to obtain a second granularity feature of the moving object to be identified includes: Processing the second image using the first sub-network model to obtain a first feature of the moving object to be identified; Processing the second image using a second sub-network model to obtain a second feature of the moving object to be identified; combining the first feature and the second feature to obtain a combined feature; Processing the combined features using an attention module to obtain a plurality of feature channels, wherein different feature channels have different weight values; The plurality of feature channels are processed using a third sub-network model to obtain second granularity features of the moving object to be identified.

15. The method according to claim 12, wherein: After determining that the moving object to be identified is a target object, the method further includes: Obtaining a historical driving trajectory of the mobile object to be identified; Processing the historical driving trajectory to obtain a driving trajectory and / or driving characteristics of the mobile object to be identified in a future period of time, wherein the driving characteristics include at least one of the following: a driving area, a parking area, and a driving time; Based on the driving trajectory and / or the driving characteristics in the future period of time, an alarm message is generated, and the mobile object to be identified is captured.

16. A storage medium comprising a stored program, wherein: When the program is running, the device where the storage medium is located is controlled to execute the target object identification method according to any one of claims 1 to 15.

17. A computing device comprising: A processor and a memory, wherein the processor is used to run a program stored in the memory, wherein the target object recognition method according to any one of claims 1 to 15 is executed when the program is run.

18. A target object recognition system comprising: processor; as well as A memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: acquiring multiple first images collected, wherein the first image contains at least one moving object; processing the first image to obtain feature information of the moving object, wherein the feature information includes: a first granularity feature and a second granularity feature; determining a target object based on the feature information of the moving object, wherein the second granularity feature is obtained by processing the first image with a second network model, and the second network model is trained using the first granularity feature; the memory is also used to provide the processor with instructions for processing the following processing steps: processing the first image with a third network model to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, processing the first image with the second network model to obtain the second granularity feature of the moving object.

19. A method for identifying a target object, comprising: Acquire a plurality of captured first images, wherein the first images contain at least one moving object; Processing the first image using a first network model to obtain a first granularity feature of the moving object; Processing the first image using a second network model to obtain a second granularity feature of the moving object, wherein the second network model includes: a first sub-network model, a second sub-network model, and a third sub-network model, the first sub-network model adopts a network structure of an 18-layer residual network with weights, the second sub-network model adopts a network structure of a 6-layer fully convolutional U-shaped neural network, and the third sub-network model is connected to the first sub-network model and the second sub-network model, and the third sub-network model adopts a network structure of a recurrent neural network; determining a target object based on the feature information of the moving object; Before using the second network model to process the first image to obtain the second granularity feature of the moving object, the method also includes: using a third network model to process the first image to obtain identification information of the moving object; comparing the processed identification information with the received identification information; if the processed identification information is consistent with the received identification information, using the second network model to process the first image to obtain the second granularity feature of the moving object.

Images