Device identity recognition, event processing method, device and storage medium

Abstract

Embodiments of the present application provide a device identity recognition method, an event processing method, a device and a storage medium. The method comprises the following steps: obtaining a to-be-recognized device picture of a target device; determining matching information of the to-be-recognized device picture and each first candidate device picture according to a device feature of the to-be-recognized device picture and a device feature of each first candidate device picture; determining a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures according to the matching information; and determining a target device identity recognition result of the target device according to a device identity recognition result of each target device picture. The image recognition technical solution provided by the embodiments of the present application can improve the device identity recognition success rate without increasing the facility cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of transportation, in particular to a device identity recognition method, an event processing method, a device and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology, intelligent highways gradually integrate information technologies such as AI (Artificial Intelligence), big data, and the Internet of Things. Through these technologies, traffic information is perceived, and traffic information services under real-time traffic data are provided. Among them, the functions of automatic perception of traffic events such as vehicle abnormal parking snapshot, red light running vehicle automatic snapshot, and vehicle overspeed automatic snapshot have been widely applied. However, the automatic and intelligent processing of illegal vehicles has not been followed up synchronously, thus greatly affecting the landing promotion and service closed loop of such intelligent perception functions. The recognition of vehicle identity (e.g., license plate number recognition) is an important basis for realizing service closed loop and intelligent law enforcement integration.

[0003] The recognition of vehicle identity is one of the most basic functions of intelligent highways and is also the most basic module for realizing vehicle digitization. Current identity recognition algorithms have high requirements for the clarity of input pictures and require the identity information on the vehicle in the input picture to be clear and visible without occlusion. Generally, only specific cameras (such as high-definition lens) can be used for shooting and recognition, and there are strict requirements for the installation angle and installation conditions, which greatly limits the promotion of related services that rely on identity recognition functions. In addition, since high-definition lens cameras are much more expensive than ordinary cameras, their distribution density is relatively low on highways (generally only one per 10 kilometers). This results in the inability to determine the identity of vehicles passing through ordinary cameras, which leads to the inability to perform intelligent disposal on vehicles sending traffic events. SUMMARY

[0004] In view of the above problems, the present application is proposed to provide a device identity recognition method, an event processing method, a device and a storage medium that solve the above problems or at least partially solve the above problems.

[0005] Therefore, in an embodiment of the present application, a device identity recognition method is provided, which includes:

[0006] obtaining a to-be-recognized device picture of a target device;

[0007] determining matching information of the to-be-recognized device picture and each first candidate device picture according to the device feature of the to-be-recognized device picture and the device feature of each first candidate device picture;

[0008] determining a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures according to the matching information.

[0009] According to the device identity recognition result of each of the multiple target device pictures, a target device identity recognition result of the target device is determined.

[0010] In another embodiment of the present application, an event processing method is provided, wherein,

[0011] After the target device is determined to have the specified event, a device picture to be recognized of the target device is obtained;

[0012] According to the device feature of the device picture to be recognized and the device feature of each of the multiple first candidate device pictures, matching information between the device picture to be recognized and each first candidate device picture is determined;

[0013] According to the matching information, multiple target device pictures belonging to the target device are determined from the multiple first candidate device pictures;

[0014] According to the device identity recognition result of each of the multiple target device pictures, a target device identity recognition result of the target device is determined.

[0015] According to the target device identity recognition result, corresponding processing is performed.

[0016] In another embodiment of the present application, an electronic device is provided. The electronic device comprises a memory and a processor, wherein,

[0017] The memory is configured to store a program.

[0018] The processor is coupled to the memory and is configured to execute the program stored in the memory to implement the method of any one of the above.

[0019] In another embodiment of the present application, a computer readable storage medium storing a computer program is provided, and the computer program is executable by a computer to implement the method of any one of the above.

[0020] In the technical solution provided in the embodiments of the present application, based on the device picture to be recognized of the target device, multiple target device pictures belonging to the target device are found through device feature matching, and then the target device identity recognition result of the target device is finally determined by comprehensively considering the device identity recognition result of each of the multiple target device pictures. That is, the technical solution provided in the embodiments of the present application does not require that the identity information in the device picture must be clear and visible. Even if the identity information is unclear or is blocked, other device pictures of the target device can be obtained through feature matching, and then the target device identity recognition result of the target device is determined based on the device identity recognition result of the other device pictures. It can be seen that the present solution can improve the device identity recognition success rate without increasing the facility cost. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the distribution of camera equipment provided in an embodiment of this application;

[0023] Figure 2 A schematic flowchart illustrating a device identification method provided in an embodiment of this application;

[0024] Figure 3 Example diagram of a device identification method provided in an embodiment of this application;

[0025] Figure 4 A flowchart illustrating an event handling method provided in an embodiment of this application;

[0026] Figure 5 A flowchart illustrating a method for handling reverse events provided in an embodiment of this application;

[0027] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] Currently, existing vehicle identification solutions rely on inference from single vehicle images and have high requirements for the quality of the input image and the specifications of the camera used to capture it, thus limiting their application. In practical use, they cannot process vehicle images taken by ordinary cameras (not checkpoint cameras), which leads to the inability to determine the identity of vehicles captured by ordinary cameras and hinders intelligent handling of vehicles involved in traffic incidents.

[0029] In practical applications, in addition to vehicle identification, it is also necessary to identify aircraft within airports and indoor robots. Taking aircraft within airports as an example, numerous cameras are deployed inside airports to capture images of aircraft. Based on these images, it is necessary to determine whether any abnormal events have occurred. If an abnormal event has occurred, the aircraft needs to be identified, and then the pilots on board need to be contacted for confirmation and relevant advice provided.

[0030] In order to be able to intelligently process all vehicles that occur traffic time, one solution is to increase the laying density of high-definition card holes on the road to improve the perception level of the vehicle identity, but this solution has high cost and it is difficult to achieve full coverage, and the feasibility is low in reality. Another solution is to use a gun-ball linkage system to remotely and accurately locate and track the target. The system locates the remote target through the gun, and the ball machine tracks the view. The ball machine enlarges the focus and captures the picture, which has the advantages of better angle, higher pixel and closer distance. However, this method can only track and locate the identity of one vehicle at the same time, and has many usage scene constraints, so the landing performance is poor.

[0031] In order to solve the above problems, the embodiment of the present application proposes a new device identity recognition method, that is, based on the to-be-recognized device picture of the target device, the target device pictures belonging to the target device are found out through device feature matching, and then the target device identity recognition result of the target device is finally determined by comprehensively combining the device identity recognition results of the plurality of target device pictures. That is, the technical scheme provided by the embodiment of the present application does not require that the identity information in the device picture must be clear and visible. Even if the identity information is unclear or blocked, other device pictures of the target device can be obtained through feature matching, and then the target device identity recognition result of the target device is determined based on the device identity recognition result of the other device pictures. It can be seen that the present solution can improve the device identity recognition success rate without increasing the facility cost, and has strong practicability.

[0032] In order to enable personnel in the technical field to better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely according to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor belong to the scope of protection of the present application.

[0033] In addition, in some of the processes described in the specification, claims, and above drawings, a plurality of operations appearing in a specific order include operations that can be executed in the order appearing in this text or in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. described herein are used to distinguish different messages, devices, modules, etc., and do not represent the order of sequence, nor do "first" and "second" represent different types.

[0034] Before introducing the device identity recognition method provided by the embodiments of the present application, the distribution of the camera devices of the expressway in the road network is introduced by taking the vehicle as an example. As shown in Figure 1 The expressway usually adopts the scheme of alternately erecting the high-definition card mouth 11 and the ordinary camera 12. The density of the high-definition card mouth is relatively low, about one per 10 km, and the distribution density of the ordinary camera is relatively high, about one per 0.8 km. The high-definition card mouth is used to capture the device identity and other detailed information; the ordinary camera is used for full-space coverage sensing.

[0035] The high-definition card mouth 11 adopts advanced photoelectric technology, image processing technology and pattern recognition technology to capture the image of each passing vehicle and identify the identity information number of the vehicle. The resolution of the vehicle image reaches 1600*1200 pixels or higher, and one photo is collected at the same time, which can clearly see the facial features of the driver and the passenger, and can also distinguish the license plate number of the vehicle with high quality, and has a high automatic recognition rate.

[0036] The ordinary camera 12 can include a gun camera, a ball camera, a fish-eye camera, etc. Usually, the full coverage of the visual sensing range of at least 800 meters can be completed by two gun cameras, one ball camera and one fish-eye camera. The ball camera is used to automatically control the observation of the key area and make a second confirmation.

[0037] In addition, all-weather radar will also be set on the expressway to collect surrounding environment information in bad weather or weak light conditions.

[0038] In actual application, the distribution of the camera device can be planned according to actual needs, which is not limited in the embodiments of the present application.

[0039] Figure 2 The flowchart of the device identity recognition method provided by an embodiment of the present application is shown. The execution subject of the method can be a client or a server. The client can be a hardware with embedded program integrated on a terminal, or an application software installed in the terminal, or a tool software embedded in the operating system of the terminal, which is not limited in the embodiments of the present application. The terminal can include any terminal device such as a camera device and a computer. The server can be a commonly used server, a cloud server or a virtual server, which is not limited in the embodiments of the present application. As shown in Figure 2 The method includes:

[0040] 201, obtaining a to-be-recognized device picture of a target device.

[0041] 202. Determine, according to the device feature of the to-be-identified device picture and the device feature of each first candidate device picture, matching information between the to-be-identified device picture and each first candidate device picture.

[0042] 203. Determine, according to the matching information, a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures.

[0043] 204. Determine, according to the device identity recognition result of each target device picture, a target device identity recognition result of the target device.

[0044] In 201, the target device refers to a movable device, for example, a vehicle, an airplane, or a robot. The target device can be any device that has traveled on a road network. When the target device is a vehicle, the road network can include urban and rural roads, expressways, and the like. When the target device is an airplane, the road network can include a plurality of roads inside an airport. When the target device is a robot, the road network can include drivable roads in the working environment of the robot, for example, the working environment of a supermarket robot is a supermarket, and the corresponding road network can include drivable areas between shelves in the supermarket. In actual applications, various devices are usually provided with identity information for identifying identity, for example, a vehicle is provided with a license plate number, an airplane is provided with a fuselage number, and a robot is provided with a robot number.

[0045] In actual applications, the to-be-identified device picture can have a low pixel. For example, when the target device is far away from a camera (usually a common camera), the to-be-identified device picture is relatively blurred. For another example, when the speed of the target device is high, the to-be-identified device picture is also relatively blurred.

[0046] In actual applications, a plurality of devices exist in a road network at the same time, and therefore, a shooting picture taken by a camera in the road network can include a plurality of devices. In order to facilitate subsequent processing, device detection can be performed on the shooting picture to obtain a device detection frame corresponding to each device, and the shooting picture is cropped according to the device detection frame corresponding to each device to obtain a device picture corresponding to each device. The device detection step can be performed by a device detection model based on deep learning. In the embodiment of the present application, all device pictures can be obtained in the above manner. In this way, when the device feature of each device is extracted, not only the representativeness of the extracted device feature can be improved, but also the calculation amount can be reduced and the calculation resources can be saved.

[0047] In the above 202, the device features of the plurality of first candidate device pictures can be extracted in advance and stored. In this way, when the device features are needed later, they can be directly obtained, which not only improves the processing efficiency, but also avoids waste of computing resources caused by repeated calculation later.

[0048] In an implementable solution, the matching information of the to-be-identified device picture and each first candidate device picture can be determined according to the similarity between the device feature of the to-be-identified device picture and the device feature of each first candidate device picture.

[0049] In an example, the similarity between the device feature of the to-be-identified device picture and the device feature of the first candidate device picture can be taken as the matching information of the to-be-identified device picture and the first candidate device picture. The matching information can be understood as a matching degree.

[0050] In another example, a similarity threshold can be obtained; if the similarity between the device feature of the to-be-identified device picture and the device feature of the first candidate device picture is greater than or equal to the similarity threshold, it is determined that the matching information of the to-be-identified device picture and the first candidate device picture is matching; if the similarity between the device feature of the to-be-identified device picture and the device feature of the first candidate device picture is less than the similarity threshold, it is determined that the matching information of the to-be-identified device picture and the first candidate device picture is not matching.

[0051] For the convenience of calculation, the device feature can be in the form of a vector, that is, the device feature includes a device feature vector. In an example, the inner product between the device feature vector of the to-be-identified device picture and the device feature vector of the first candidate device picture can be taken as the similarity between the two.

[0052] In order to improve the rationality of the matching information, the device feature of the to-be-identified device picture and the device feature of each of the plurality of first candidate device pictures can be extracted by the same target feature extraction model. The training process of the target feature extraction model will be described in detail in the following embodiments.

[0053] In the above 203, the first candidate device picture that matches or has a high matching degree with the to-be-identified device picture in the plurality of first candidate device pictures can be considered as the plurality of target device pictures of the target device.

[0054] When the matching information includes the matching degree, the plurality of first candidate device pictures can be sorted in descending order according to the matching degree, and the first candidate device pictures in the top preset number of the sorted list can be taken as the plurality of target device pictures of the target device, or the first candidate device picture whose matching degree is greater than or equal to a preset matching degree threshold can be taken as the target device picture of the target device.

[0055] In actual applications, the target device picture can be input into the trained device identity recognition model to output a device identity recognition result from the device identity recognition model.

[0056] Generally, the device identity recognition model also outputs a device identity recognition confidence. The greater the device identity recognition confidence, the greater the accuracy of the corresponding device identity recognition result.

[0057] Therefore, in an example, the device identity recognition confidence of each of the plurality of target device pictures can be obtained, and the device identity recognition result of the target device picture with the greatest device identity recognition confidence can be taken as the target device identity recognition result of the target device.

[0058] In the technical solution provided by the embodiments of the present application, based on the to-be-recognized device picture of the target device, a plurality of target device pictures belonging to the target device are found out through device feature matching, and then the target device identity recognition result of the target device is finally determined by comprehensively considering the device identity recognition results of the plurality of target device pictures. That is, the technical solution provided by the embodiments of the present application does not require that the identity information in the device picture must be clear and visible. Even if the identity information is unclear or blocked, other device pictures of the target device can be obtained through feature matching, and then the target device identity recognition result of the target device is determined based on the device identity recognition result of the other device pictures. It can be seen that the present solution can improve the device identity recognition success rate without increasing the facility cost, and has strong practicability.

[0059] In actual applications, the plurality of first candidate device pictures can be obtained by a plurality of camera devices distributed in a road network.

[0060] In actual applications, the plurality of first candidate device pictures can be all backup device pictures in a target device database. The target device database is created according to the shooting results of a plurality of camera devices distributed in a road network, and the target device database stores a plurality of backup device pictures. The shooting results include a plurality of shooting pictures. The device detection frame can be obtained by performing device detection on the shooting pictures, and the device picture can be obtained by cropping the shooting pictures according to the device detection frame, and then stored in the target device database as a backup device picture. The plurality of camera devices can include a high-definition lens and / or a general camera.

[0061] In order to improve the accuracy of subsequent device identity recognition, the clarity of the backup device pictures stored in the target database meets the preset requirement. That is, only the device pictures with the clarity meeting the preset requirement are stored in the target device database. Whether the clarity of the device picture meets the preset requirement can be determined according to the size of the device identity recognition confidence of the device picture. For example, when the device identity recognition confidence of the device picture is greater than or equal to a first device identity recognition confidence threshold (for example, 60%), it is determined that the clarity of the device picture meets the preset requirement; otherwise, it is determined that the clarity of the device picture does not meet the preset requirement. Generally speaking, the higher the clarity of the device picture, the higher the device identity recognition confidence corresponding to the device identity recognition result obtained by using the device identity recognition model to perform device identity recognition on the device picture.

[0062] Taking a vehicle as an example, a plurality of camera devices in the road network will shoot a large number of shooting pictures every day, that is, the number of backup device pictures in the target device database is large. If a full-amount search is performed in the target device database without any purpose, not only the search accuracy cannot be guaranteed, but also the challenge to the search engine is large. Therefore, the above method can further include:

[0063] 205. Determine a first search range according to the shooting parameters of the to-be-recognized device picture.

[0064] The shooting parameters include the shooting time and / or the shooting location.

[0065] 206. Extract the plurality of first candidate device pictures from the target device database according to the first search range.

[0066] In the above 205, when the shooting parameters include the shooting time, the first search time range can be determined according to the shooting time, and the first search time range is taken as the first search range. The shooting time can be the midpoint time of the first search time range. Assuming that the shooting time is T, the first search time range can be [T-t1, T+t1], wherein the value of t1 can be set according to actual needs, and the embodiments of the present application do not make specific limitation thereon. For example, when the target device is a vehicle, the value of t1 can be 0.5 h.

[0067] When the photographing parameter comprises a photographing location, the first search space range can be determined according to the photographing location; and the first search space range is taken as the first search range. The first search range can be centered on the photographing location. Assuming that the upper limit of the vehicle speed is v km / h, the preset driving time is t2, and the redundancy coefficient is a, then the path length that the target device can travel within the preset driving time t2 is L1 = v*t4*a km. Therefore, when actually searching, only the circular search space range with the photographing location of the picture of the device to be identified as the center and a radius of L1 km needs to be considered. The t2 can be equal to or different from the t1.

[0068] Considering the complexity of the road network in reality, the first search space range can be determined according to the photographing location and the road network information. Specifically, a plurality of reachable roads that can reach the photographing location can be determined according to the road network information; the first target road segment of each of the plurality of reachable roads is determined according to the path length L1 that the target device can travel; and the first target road segments of the plurality of reachable roads are taken as the first search space range. One end of the first target road segment is the photographing location.

[0069] Taking the target device as a vehicle as an example, since the roads in reality are usually two-way lanes, the driving direction of the target device on a road is generally fixed. Therefore, in actual application, the first search space range can be further compressed in combination with the driving direction of the target device. Specifically, the driving direction of the target device can be determined according to the picture of the device to be identified; and the first search space range is determined according to the photographing location, the road network information and the driving direction. Specifically, the first target road segment of each of the plurality of reachable roads can be determined according to the above method; the target lane is determined from each first target road segment according to the driving direction; the driving direction of the target lane is consistent with the driving direction of the target device; and the target lane of the first target road segment of each of the plurality of reachable roads is taken as the first search space range.

[0070] When the photographing parameter comprises a photographing time and a photographing location, the first search range can comprise the first search time range and the first search space range.

[0071] In 206, a plurality of first candidate device pictures whose photographing parameters belong to the first search range can be extracted from the target device database.

[0072] Specifically, when the photographing reference comprises a photographing time, a plurality of first candidate device pictures whose photographing times are located in the first search time range can be extracted from the target device database.

[0073] When the shooting parameter comprises a shooting location, a plurality of first candidate device pictures with the shooting location in the first search space range can be extracted from the target device database.

[0074] When the shooting parameter comprises a shooting time and a shooting location, a plurality of first candidate device pictures with the shooting time in the first search time range and the shooting location in the first search space range can be determined from the target device database.

[0075] In the embodiments of the present application, since the target device appears in the first search range with the maximum probability, the search efficiency can be effectively improved, and the search accuracy can also be improved.

[0076] In actual applications, the clarity of the to-be-identified device picture is likely to be poor, and the target device database stores backup device pictures with good clarity. This results in a large difference between the device features of the to-be-identified device picture and the device features of the device pictures in the target device database, which is not conducive to subsequent matching or searching. Therefore, the step of determining the matching information between the to-be-identified device picture and each first candidate device picture according to the device features of the to-be-identified device picture and the device features of each first candidate device picture in 202 can be implemented as follows:

[0077] 2021. Determine a second search range according to the shooting parameter of the to-be-identified device picture.

[0078] The second search range is smaller than the first search range.

[0079] 2022. Extract a plurality of first backup device pictures from the target device database according to the second search range.

[0080] 2023. Determine a replacement device picture for replacing the to-be-identified device picture from the plurality of first backup device pictures according to the similarity between the device features of the to-be-identified device picture and the device features of each first backup device picture in the plurality of first backup device pictures.

[0081] 2024. Determine the matching information between the to-be-identified device picture and each first candidate device picture according to the similarity between the device features of the replacement device picture and the device features of each first candidate device picture.

[0082] In the above 2021, when the shooting parameter includes a shooting time, a second search time range can be determined according to the shooting time; and the second search time range is taken as the second search range. The shooting time can be a midpoint time of the second search time range. Assuming that the shooting time is T, the second search time range can be [T-t3, T+t3], where the value of t3 can be set according to actual needs, and the embodiments of the present application do not make specific limitations thereon. Wherein, t3 is less than t1. For example: the value of t3 is 3min or 5min.

[0083] When the shooting parameter includes a shooting location, a second search space range can be determined according to the shooting location; and the second search space range is taken as the second search range. Wherein, the second search range can be centered on the shooting location. Assuming that the upper limit of the vehicle speed is v km / h, the preset driving time is t4, and the redundancy coefficient is a, then within the preset driving time t4, the path length that the target device can travel is L2=v*t4*a km. Then, only a circular search space range with the shooting location of the picture of the device to be identified as the center and a radius of L2 km needs to be considered in the actual search. Wherein, t4 is less than t2.

[0084] Considering the complexity of the road network in reality, the second search space range can be determined according to the shooting location and the road network information. Specifically, a plurality of reachable roads that can reach the shooting location can be determined according to the road network information; a second target road segment on each reachable road is determined according to the calculated path length L2 that the target device can travel; and the second target road segments corresponding to the plurality of reachable roads are taken as the second search space range. One end of the second target road segment is the shooting location.

[0085] Taking the target device as a vehicle as an example, since the roads in reality are usually two-way lanes, the driving direction of the target device on a certain road is generally fixed and unchangeable. Therefore, in actual application, the second search space range can be further compressed in combination with the driving direction of the target device. Specifically, the driving direction of the target device can be determined according to the picture of the device to be identified; and the second search space range is determined according to the shooting location, the road network information and the driving direction. Specifically, the second target road segments corresponding to the plurality of reachable roads can be first determined according to the above method; a target lane is determined from each second target road segment according to the driving direction; wherein, the drivable direction of the target lane is consistent with the driving direction of the target device; and the target lanes of the second target road segments corresponding to the plurality of reachable roads are taken as the second search space range.

[0086] In the above 2022, a plurality of first backup device pictures in which the shooting parameter belongs to the second search range can be extracted from the target device database.

[0087] Specifically, when the photographing reference includes the photographing time, the first backup device pictures with photographing time within the second search time range can be extracted from the target device database.

[0088] When the photographing parameter includes the photographing location, the first backup device pictures with photographing location within the second search space range can be extracted from the target device database.

[0089] When the photographing parameter includes the photographing time and the photographing location, the first backup device pictures with photographing time within the second search time range and photographing location within the second search space range can be determined from the target device database.

[0090] In 2023, the first backup device picture with the largest similarity can be used as the replacement device picture for replacing the to-be-identified device picture.

[0091] In 2024, in an implementable solution, the similarity between the device feature of the replacement device picture and the device feature of each first candidate device picture can be used as the matching degree between the to-be-identified device picture and each first candidate device picture. The matching information includes the matching degree.

[0092] In actual application, the method can further obtain a device identity recognition confidence corresponding to the replacement device picture; if the device identity recognition confidence is greater than or equal to a second device identity recognition confidence threshold (for example, 95%), the device identity recognition result corresponding to the replacement device picture can be directly used as the target device identity recognition result of the target device; otherwise, the step 2024 and subsequent steps are executed to determine the target device identity recognition result of the target device.

[0093] In this embodiment, the replacement device picture for replacing the to-be-identified device picture is obtained through fuzzy matching, and then accurate matching is performed based on the device feature of the replacement device picture. The device feature of the replacement device picture is similar to the device feature distribution of the backup device pictures in the target device database, which helps to improve the subsequent search accuracy.

[0094] In an example, in addition to storing the backup device pictures, the target device database can also store one or more of the device feature, the photographing time, the photographing location, the device identity recognition result and the device identity recognition confidence, and the device attribute of each backup device picture. The device attribute can include the device color, the device brand, the device model, and the like.

[0095] To further improve search accuracy, a front-facing device database and a back-facing device database can be established. The front-facing device database displays images of the device's front side, while the back-facing device database displays images of the device's back side. Therefore, the above method may also include:

[0096] 207. Determine whether the image of the device to be identified is a frontal image of the device, and obtain the determination result.

[0097] 208. Based on the determination result, determine the target device database from multiple device databases.

[0098] In one feasible solution, such as Figure 3 As shown, the device image 31 to be identified can be input into the trained front / back recognition model 32 to obtain the front / back recognition result. The front / back recognition model is also a classification model. The specific implementation and training process of the classification model can be found in existing technologies and will not be detailed here. If the device image 31 to be identified is a front-facing device image, the front-facing device database 33 from multiple device databases can be used as the target device database. If the device image 31 to be identified is not a front-facing device image but a back-facing device image, the back-facing device database 34 from multiple device databases can be used as the target device database.

[0099] Furthermore, based on the above determination results, a target feature extraction module can be identified from the front feature extraction model and the back feature extraction model. If the device image to be identified is a front-facing device image, the front feature extraction model can be used as the target feature extraction module; this target feature extraction module will extract the device features of the device image to be identified. If the device image to be identified is not a front-facing device image, but a back-facing device image, the back feature extraction model can be used as the target feature extraction module; this target feature extraction module will extract the device features of the device image to be identified.

[0100] The front feature extraction model can be trained using front sample device images, and the back feature extraction model can be trained using negative sample device images. The specific training process will be described in detail in the following embodiments.

[0101] The equipment features of the spare equipment images in the aforementioned front-facing equipment database are extracted using a front-facing feature extraction model; the equipment features of the spare equipment images in the aforementioned back-facing equipment database are extracted using a back-facing feature extraction model. For example... Figure 3As shown, in the process of creating the database, the front-back identification model 32 can be used to identify the front and back of the device picture 37; if the device picture 37 is a front device picture, the front feature model 35 is used to extract the features of the device picture 37 to obtain the device features of the device picture 37; the device picture 37 and its device features are stored in the front device database 33; if the device picture 37 is a back device picture, the back feature model 36 is used to extract the features of the device picture 37 to obtain the device features of the device picture 37; the device picture 37 and its device features are stored in the back device database 34.

[0102] Further, the "determining, according to the matching information, a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures" in the above 203 can be implemented by the following steps:

[0103] 2031. Determining, according to the matching information, a plurality of second candidate device pictures from the plurality of first candidate device pictures.

[0104] 2032. Screening, according to reference information, a plurality of target device pictures belonging to the target device from the plurality of second candidate device pictures.

[0105] The reference information includes: the device identity recognition confidence of each of the plurality of second candidate device pictures and / or the matching of the device attributes of each of the plurality of second candidate device pictures with the device attributes of the target device.

[0106] In the above 2031, the matching information can include a matching degree, and according to the matching degree, the plurality of first candidate device pictures are sorted in descending order, and the first candidate device pictures in the top preset number are taken as the plurality of second candidate device pictures.

[0107] In the above 2032, the plurality of third candidate device pictures in the plurality of second candidate device pictures whose device identity recognition confidence is greater than or equal to a third device identity recognition confidence threshold (for example: 90%) can be taken as the plurality of target device pictures belonging to the target device; or, the plurality of third candidate device pictures in the plurality of second candidate device pictures whose device attributes match the device attributes of the target device can be taken as the plurality of target device pictures belonging to the target device; or, the plurality of third candidate device pictures in the plurality of second candidate device pictures whose device identity recognition confidence is greater than or equal to a second device identity recognition confidence threshold and whose device attributes match the device attributes of the target device can be taken as the plurality of target device pictures belonging to the target device.

[0108] Optionally, the device identity recognition result comprises a string used for identifying the device identity. Taking a vehicle as an example, the string is a license plate number. The step of determining the target device identity recognition result of the target device according to the device identity recognition result of each of the plurality of target device pictures in 204 can be implemented as follows:

[0109] 2041. Determine the character at the n th position in the string used for identifying the device identity in each of the plurality of target device pictures.

[0110] 2042. Take the character appearing most frequently at the n th position as the target device identity information at the n th position.

[0111] The value of n ranges from 1 to N, where N is the length of the string. The target device identity recognition result comprises the target device identity information at the n th position.

[0112] In this embodiment, the device identity recognition results of the plurality of target device pictures are fused through a voting mechanism. Through information fusion, the accuracy of the identity information recognition can be maximized. For example, in some scenarios, the device identity information can only be correctly identified at some fields at different points, and the identity information recognition result of a single point is partially incorrect. Through cross-point result fusion, the local recognition error can be eliminated.

[0113] The training process of the target feature extraction model will be described below.

[0114] 209. Obtain a triple sample used for training.

[0115] The triple sample comprises a first sample device picture and a second sample device picture of a first sample device, and a third sample device picture of a second sample device.

[0116] 210. Take the triple sample as the input of the target feature extraction model, and obtain the device feature of each sample device picture in the triple sample.

[0117] 211. Determine a triple loss according to the device feature of each sample device picture in the triple sample.

[0118] 212. Optimize the target feature extraction model according to the triple loss.

[0119] In 211, a first distance between the device feature of the first sample device picture and the device feature of the second sample device picture can be calculated; a second distance between the device feature of the first sample device picture and the device feature of the third sample device picture can be calculated; and a triplet loss can be determined according to the first distance and the second distance. The specific determination manner of the triplet loss can refer to the prior art, and will not be described in detail herein.

[0120] In 212, the optimization of the target feature extraction model aims to make the second distance greater than the first distance.

[0121] Specifically, the model can be optimized by using a gradient back propagation algorithm, and the specific optimization process can refer to the prior art, and will not be described in detail herein.

[0122] In order to ensure that the target feature extraction model can extract the device itself feature, the device identity region in each sample device picture in the triplet sample can be occluded, so that the target feature extraction model can avoid focusing attention on the device identity region and ignoring the extraction of the device itself feature. The device identity region refers to a region that displays device identity information, for example, a license plate region.

[0123] During training, the first sample device and the second sample device involved in the triplet sample can be two devices of the same color, model and brand, for example, two vehicles of the same color, model and brand. That is, the triplet sample is a difficult triplet sample, which helps the target device feature extraction model to learn to extract more subtle features. Taking a vehicle as an example, the device internal layout, decoration and other features displayed through the vehicle window, and the prompt picture (for example, there is a baby in the vehicle, and an intern) attached outside the vehicle can be learned.

[0124] Further, in 212, the target feature extraction model can be optimized according to the triplet loss, which can be achieved by the following steps:

[0125] 2121. According to the device feature of the first sample device picture of the first sample device, the predicted device identity of the first sample device is determined through a classification network.

[0126] 2122. According to the predicted device identity and the real device identity of the first sample device, a classification loss is determined.

[0127] 2123. According to the classification loss and the triplet loss, the target feature extraction model is optimized.

[0128] In 2121, the classification network predicts a probability that the device identity of the first sample device belongs to a plurality of candidate device identities according to the device features of the device picture of the first sample device; and the candidate device identity with the highest probability is taken as the predicted device identity of the first sample device.

[0129] The plurality of candidate device identities can be determined according to a training set.

[0130] In 2122, a cross-entropy loss function can be used to calculate the classification loss, and a specific implementation of the cross-entropy loss function can be selected or designed according to actual needs, which is not limited in the embodiments of the present application.

[0131] In 2123, in an example, the target feature extraction model can be optimized according to the sum of the classification loss and the triplet loss.

[0132] In an implementable solution, the target feature extraction model can include a backbone network and a global average pooling layer. The classification network can include a fully connected layer. The backbone network of the model can use a commonly used backbone of a picture classification network.

[0133] Figure 4 A flowchart of an event processing method provided by another embodiment of the present application is shown. The execution subject of the method can be a client or a server. The client can be a hardware with an embedded program integrated in a terminal, an application software installed in the terminal, a tool software embedded in the operating system of the terminal, etc., which are not limited in the embodiments of the present application. The terminal can include any terminal device such as a camera device and a computer distributed in a road network. The server can be a commonly used server, a cloud server or a virtual server, which are not limited in the embodiments of the present application. As shown in the figure, the method includes: Figure 4

[0134] 401. After determining that a specified event occurs to a target device, a device picture to be identified of the target device is obtained.

[0135] 402. According to the device features of the device picture to be identified and the device features of each of a plurality of first candidate device pictures, matching information between the device picture to be identified and each of the first candidate device pictures is determined.

[0136] 403. According to the matching information, a plurality of target device pictures belonging to the target device are determined from the plurality of first candidate device pictures.

[0137] 404. According to the device identity recognition results of each of the plurality of target device pictures, a target device identity recognition result of the target device is determined.

[0138] ​405、According to the target device identity recognition result, corresponding processing is performed.

[0139] Taking a vehicle or an airplane as an example of the target device, the specified event refers to traffic time, including device reverse, device speed anomaly, device stop, device illegal lane change, emergency lane occupation, traffic accident, etc.

[0140] Taking a robot as an example of the target device, the specified event can be an event that requires human intervention, such as a robot hitting a person event, a hitting a shelf event, etc. The specified event can be set according to actual needs, and the embodiments of the present application do not make specific limitations.

[0141] Taking a vehicle as an example of the target device, the above-mentioned corresponding processing can include: deduction processing, fine processing, warning processing, obtaining the phone number of the vehicle owner, etc.

[0142] For example, taking a vehicle as an example of the target device, when a traffic accident occurs to the target device, the phone number of the vehicle owner can be obtained, and then the vehicle owner can be contacted to understand the details of the traffic accident, so as to make good rescue preparation.

[0143] The specific implementation of the above steps 401 to 404 can refer to the corresponding content in the above embodiments, which will not be repeated here.

[0144] It should be noted that the contents of the steps in the method provided by the embodiments of the present application which are not fully described can refer to the corresponding contents in the above embodiments, which will not be repeated here. In addition, the method provided by the embodiments of the present application can include other parts or all steps in the above embodiments in addition to the above steps, which can refer to the corresponding contents of the above embodiments, which will not be repeated here.

[0145] The specific implementation of the above steps 401 to 404 can refer to the corresponding content in the above embodiments, which will not be repeated here. Figure 5 The event processing method provided by the embodiments of the present application will be introduced as follows:

[0146] As shown in Figure 5 The camera device 51 on the highway determines the reverse vehicle through the vehicle picture of the reverse vehicle, and sends the vehicle picture to be identified 52 of the reverse vehicle to the server 53. After the server 53 receives the vehicle picture to be identified 52 of the reverse vehicle, the following steps are performed:

[0147] S1, according to the vehicle features of the vehicle picture to be identified of the reverse vehicle and the vehicle features of each of the plurality of first candidate vehicle pictures, determine the matching information of the vehicle picture to be identified and each first candidate vehicle picture.

[0148] S2, according to the matching information, determine a plurality of target vehicle pictures belonging to the reverse vehicle from the plurality of first candidate vehicle pictures.

[0149] S3, determining the target vehicle identity recognition result of the reverse vehicle according to the vehicle identity recognition results of the plurality of target vehicle pictures respectively.

[0150] S4, contacting the vehicle owner by phone according to the target vehicle identity recognition result.

[0151] In practical applications, the intelligent robot can contact the vehicle owner by phone to understand the situation and give suggestions.

[0152] In summary, the technical scheme provided by the embodiments of the present application fuses and reasons the identity information of the same vehicle under a plurality of cameras based on a cross-camera vehicle feature matching algorithm, supplements the vehicle identity of the vehicle photographed by an ordinary camera which cannot recognize the license plate number with the high-quality camera with high-trust vehicle identity information, and further realizes the global vehicle identity perception. The camera used for event and accident detection generally does not have license plate recognition capability or has weak recognition capability, and in some cases, the event and accident occurs far away from the shooting camera and cannot capture the identity information of the event and accident vehicle. Therefore, the vehicle identity reasoning method can be used in the specified event detection system to infer the identity information of the event and accident vehicle, further help the road manager to determine the vehicle owner identity as soon as possible, speed up the processing of the event and accident and reduce the road congestion caused by the event and accident. The scheme can be integrated into the event detection system to help the event detection function closed loop landing. The scheme does not require clear license plate input pictures, and the license plate can be recognized through identity reasoning, which is the main problem to be solved by the patent; the scheme comprehensively utilizes the data under a plurality of cameras, and the information between the cameras can be effectively linked based on the image feature matching of the vehicle. Through information fusion, the accuracy of license plate recognition can be maximized. It can be seen that based on the current distribution mode of camera points on the highway, the scheme proposes a cross-camera multi-source information fusion vehicle identity reasoning scheme, realizes global vehicle identity reasoning, creates a vehicle "magnifying glass" for a large number of cameras without vehicle identity recognition function, realizes ordinary camera "high-definition hole", and can accurately infer the searched license plate number based on the matching vehicle license plate information in real time.

[0153] Figure 6 The structure schematic diagram of the electronic device provided by an embodiment of the present application is shown. As shown in Figure 6As shown, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 can be configured to store various data to support operations on the electronic device. Examples of these data include instructions for any application or method operating on the electronic device. The memory 1101 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0154] The memory 1101 is configured to store programs.

[0155] The processor 1102 is coupled to the memory 1101 and is configured to execute the programs stored in the memory 1101 to implement the device identity recognition method and the event processing method provided by the above method embodiments.

[0156] Further, as shown in Figure 6 The electronic device further includes a communication component 1103, a display 1104, a power supply component 1105, an audio component 1106, and other components. Figure 6 Some components are only schematically shown in the electronic device, and it does not mean that the electronic device only includes Figure 6 the components shown.

[0157] Correspondingly, the embodiments of the present application further provide a computer readable storage medium storing a computer program, which can implement the steps or functions of the device identity recognition method and the event processing method provided by the above method embodiments when the computer program is executed by a computer.

[0158] The device embodiments described above are only schematic and the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. Those skilled in the art can understand and implement it without creative labor.

[0159] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the various embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some parts of the embodiment. The above various embodiments require the combination of CPU and GPU two development environments to achieve better speed and performance.

[0160] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method of identifying a device identity, wherein, The method comprises the following steps: obtaining a to-be-identified device picture of a target device, the to-be-identified device picture being obtained by a certain shooting device arranged in a road network, the target device being a movable device, and the target device being provided with identity information for identifying identity; determining matching information of the to-be-identified device picture and each first candidate device picture according to device features of the to-be-identified device picture and device features of each first candidate device picture, the first candidate device pictures being obtained by a plurality of shooting devices distributed in the road network, and a device identity recognition confidence of each first candidate device picture being greater than or equal to a preset threshold; determining a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures according to the matching information; determining a target device identity recognition result of the target device according to device identity recognition results of the plurality of target device pictures.

2. The method of claim 1, wherein, Further comprising: determining a first search range according to a shooting parameter of the to-be-identified device picture; the shooting parameter comprising a shooting time and / or a shooting location; extracting the plurality of first candidate device pictures from a target device database according to the first search range, the target device database being created according to shooting results of a plurality of shooting devices distributed in the road network, and the target device database storing a plurality of backup device pictures.

3. The method of claim 2, wherein, the shooting parameter comprising a shooting time and a shooting location; determining a first search range according to a shooting parameter of the to-be-identified device picture, comprising: determining a first search space range according to the shooting location; determining a first search time range according to the shooting time; the first search range comprising the first search time range and the first search space range.

4. The method of claim 3, wherein, extracting the plurality of first candidate device pictures from the target device database according to the first search range, comprising: determining, from the target device database, a plurality of first candidate device pictures whose shooting time is within the first search time range and whose shooting location is within the first search space range.

5. The method of claim 4, wherein, determining matching information of the to-be-identified device picture and each first candidate device picture according to device features of the to-be-identified device picture and device features of each first candidate device picture, comprising: determining a second search range according to a shooting parameter of the to-be-identified device picture, the second search range being smaller than the first search range; extracting a plurality of first backup device pictures from the target device database according to the second search range; determining a replacement device picture for replacing the to-be-identified device picture from the plurality of first backup device pictures according to a similarity degree between the device features of the to-be-identified device picture and device features of each first backup device picture; determining matching information of the to-be-identified device picture and each first candidate device picture according to a similarity degree between the device features of the replacement device picture and device features of each first candidate device picture.

6. The method of any one of claims 2 to 5, wherein, Further comprising: determining whether the to-be-identified device picture belongs to a front device picture, to obtain a determination result; According to the determination result, the target device database is determined from the plurality of device databases.

7. The method of any one of claims 1 to 5, wherein, According to the matching information, a plurality of target device pictures belonging to the target device are determined from the plurality of first candidate device pictures, including: According to the matching information, a plurality of second candidate device pictures are determined from the plurality of first candidate device pictures; According to the reference information, a plurality of target device pictures belonging to the target device are screened from the plurality of second candidate device pictures; The reference information includes: a device identity recognition confidence corresponding to each of the plurality of second candidate device pictures and / or a matching condition of a device attribute corresponding to each of the plurality of second candidate device pictures and a device attribute of the target device.

8. The method of any one of claims 1 to 5, wherein, The device identity recognition result includes: a string for identifying device identity; According to the device identity recognition result of each of the plurality of target device pictures, a target device identity recognition result of the target device is determined, including: The character at the nth position in the string for identifying device identity of each of the plurality of target device pictures is determined; The character appearing most frequently at the nth position is taken as the target device identity information at the nth position.

9. The method of any one of claims 1 to 5, wherein, The device features of the to-be-recognized device picture and the device features of each first candidate device picture are extracted by a trained target feature extraction model; The training process of the target feature extraction model includes: Obtaining a triple sample for training; the triple sample includes: a first sample device picture and a second sample device picture of a first sample device, and a third sample device picture of a second sample device; The triple sample is taken as the input of the target feature extraction model to obtain the device features of each sample device picture in the triple sample; According to the device features of each sample device picture in the triple sample, a triple loss is determined; According to the triple loss, the target feature extraction model is optimized.

10. The method of claim 9, wherein, The device identity region in each sample device picture in the triple sample is blocked.

11. The method of claim 9, wherein, According to the triple loss, the target feature extraction model is optimized, including: According to the device features of the first sample device picture of the first sample device, the predicted device identity of the first sample device is determined through a classification network; According to the predicted device identity and the real device identity of the first sample device, a classification loss is determined; According to the classification loss and the triple loss, the target feature extraction model is optimized.

12. The method of any one of claims 1 to 5, wherein, The target device includes: a vehicle, an airplane or a robot.

13. An event processing method, wherein, After determining that a target device occurs a specified event, a to-be-recognized device picture of the target device is obtained, the to-be-recognized device picture is obtained by a shooting device arranged in a road network; the target device is a movable device; the target device is provided with identity information for identifying identity; According to the device feature of the to-be-identified device picture and the device feature of each first candidate device picture, determine matching information of the to-be-identified device picture and each first candidate device picture, the first candidate device pictures are obtained by a plurality of shooting devices distributed in the road network, and the device identity recognition confidence of each first candidate device picture is greater than or equal to a preset threshold; According to the matching information, determine a plurality of target device pictures belonging to the target device from the plurality of first candidate device pictures; According to the device identity recognition result of each target device picture, determine a target device identity recognition result of the target device; According to the target device identity recognition result, perform corresponding processing.

14. An electronic device, comprising: Comprise: Memory and processor, wherein, The memory is used to store programs; The processor is coupled with the memory, and is used to execute the programs stored in the memory to realize the method in any one of claims 1 to 13.

15. A computer readable storage medium storing a computer program, wherein, The computer program is executed by the computer to realize the method in any one of claims 1 to 13. The computer program is executed by the computer to realize the method in any one of claims 1 to 13.

Images