A monitoring method and system based on face recognition and license plate recognition

Abstract

The present invention discloses a monitoring method and system based on face recognition and license plate recognition, which relates to the technical field of driver monitoring, including acquiring images and video streams, performing data processing on the acquired data; performing target object recognition to screen out pedestrians, and recognizing license plates and faces; and performing monitoring and analysis in combination with the recognition results of license plates and faces. The monitoring method based on face recognition and license plate recognition provided by the present invention can automatically determine whether a vehicle has problems such as overloading and illegal driving by combining license plate recognition, face recognition and driver behavior analysis, and provide real-time feedback to management personnel through an intelligent early warning system, thereby reducing human intervention and misjudgment. The judgment is made in combination with vehicle type and driver information, and the analysis is more comprehensive, efficient and accurate to assist management personnel. The present invention achieves better results in both automation and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of driver monitoring, and in particular to a monitoring method and system based on face recognition and license plate recognition. Background Art

[0002] With the rapid development of intelligent technology, image recognition technology has been widely used in many fields, especially in traffic monitoring and security management. Face recognition and license plate recognition technology based on computer vision has become an indispensable part of intelligent transportation systems. In recent years, the rise of deep learning and convolutional neural networks (CNN) has greatly improved the analysis accuracy and efficiency of images and video streams. Existing license plate recognition technology usually uses high-precision image processing algorithms such as YOLOv5 and SSD to quickly locate the license plate area and perform character recognition; face recognition technology extracts facial features through deep neural networks (such as Siamese networks and ResNet) and combines contrast loss functions for high-precision identity authentication. With the improvement of computing power, the real-time and accuracy of image and video data processing have been significantly improved, becoming one of the core technologies for traffic monitoring, vehicle management and security review.

[0003] Although the existing license plate recognition and face recognition technologies have achieved certain results in the field of traffic monitoring, there are still many shortcomings in practical applications, especially in terms of multi-dimensional information combination, real-time, flexibility and adaptability in complex scenarios. First, most of the existing technologies are based on a single information source for matching. For example, license plate recognition only relies on the license plate area in the image for recognition, while face recognition only uses the image obtained by the camera for identity authentication. This matching method based on a single information source cannot effectively cope with complex scenarios. For example, when the driver's identity is inconsistent with the owner's identity, how to dynamically verify whether the driver is driving legally and respond quickly. Secondly, the existing systems often lack a deep understanding of real-time data and the combination of multi-dimensional information, resulting in the inability to effectively handle complex interactions in multi-camera monitoring, dynamic authorization and driver behavior analysis. For example, in a multi-camera monitoring environment, the existing technology can only recognize license plates or faces through data captured by a single camera, ignoring the time synchronization and perspective changes between different cameras, and cannot accurately identify and combine behavioral data from multiple perspectives. In addition, the existing technology has weak support for authorization changes and behavior pattern analysis, and lacks the ability to perform real-time dynamic analysis of driver behavior and identity in multi-role scenarios (such as taxis, shared vehicles, etc.). Finally, existing technologies often lack intelligent behavior analysis and real-time response mechanisms when dealing with complex scenarios such as overloading, illegal driving, and unauthorized driving. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: the existing driving monitoring method combining license plate recognition and face recognition has the problem of single information source recognition, lacks the ability to process multi-camera synchronous data, cannot effectively handle the matching relationship between drivers and car owners in complex scenes, and how to analyze driver behavior in real time, dynamically verify identity and judge illegal behavior.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a monitoring method based on face recognition and license plate recognition, comprising acquiring images and video streams, and processing the acquired data. Performing target object recognition to screen out pedestrians, and recognizing license plates and faces. Monitoring and analysis are performed in combination with the recognition results of license plates and faces.

[0007] As a preferred solution of the monitoring method based on face recognition and license plate recognition described in the present invention, the acquisition of images and video streams includes the front-end camera transmitting the video stream to the back-end server through the RTSP protocol, the server decoding the video stream, and extracting the image data of each frame therefrom.

[0008] After the video stream is decoded, the server converts each frame in the video into a static image.

[0009] When there are multiple cameras monitoring the same area, the video stream of each camera will be decoded into independent image frames, marked with timestamps, and each set of image data belongs to the same moment.

[0010] As a preferred solution of the monitoring method based on face recognition and license plate recognition described in the present invention, the data processing of the acquired data includes processing the image after acquiring the image data, and using histogram equalization to improve the brightness contrast of the image when the image environment is rainy, night or cloudy.

[0011] Use Gaussian filtering to reduce noise in the image, and the Gaussian filter kernel size is 5x5.

[0012] After removing the noise, the Canny edge detection algorithm is used to extract the obvious edge features in the image. The high threshold is set to 150 and the low threshold is set to 50. Edge pixels below the low threshold are identified as non-edges and ignored, and those between the high threshold and the low threshold are identified as weak edges. Weak edges are retained only when connected to strong edges, otherwise they are discarded. Edge pixels above the high threshold are retained. After completing the edge construction, the area containing the vehicle is preliminarily screened out.

[0013] The target objects are pedestrians and vehicles.

[0014] As a preferred solution of the monitoring method based on face recognition and license plate recognition described in the present invention, the target object recognition and screening of pedestrians includes using YOLOv5 to perform target detection on the target object. YOLOv5 scans and identifies the category of the target object in the image, including vehicles and pedestrians, and marks the position of the vehicle in the image. The position of each target is represented by a bounding box.

[0015] When YOLOv5 identifies a vehicle, it further detects whether the vehicle contains a license plate area.

[0016] License plate area detection uses the YOLOv5 model to identify the license plate area , the coordinates are the bounding box coordinates, expressed as:

[0017]

[0018] in, Represents the coordinates of the four corners of the license plate bounding box.

[0019] As a preferred embodiment of the monitoring method based on face recognition and license plate recognition described in the present invention, the recognition of license plates and faces includes generating character images using a generative adversarial network, and the images generated by GAN The license plate characters are extracted to obtain the enhanced character area, which is expressed as:

[0020]

[0021] in, Represents a license plate image.

[0022] The convolutional neural network with attention mechanism is used to recognize license plate characters, which is expressed as:

[0023]

[0024]

[0025] in, Indicates Character area, is the recognition result of each character, and n is the number of license plate characters.

[0026] As a preferred solution of the monitoring method based on face recognition and license plate recognition described in the present invention, the recognition of license plates and faces also includes first extracting face features by jointly using a convolutional neural network and a Siamese network, expressed as

[0027]

[0028] in, and For the two initial input face images, when only one camera obtains data, two consecutive frames of images are selected for calculation. When there are multiple cameras, a group of images at the same time are selected for calculation. For the face images stored in the database and the extracted input image The features are used to calculate the Euclidean distance and converted into similarity through the Sigmoid function, which is expressed as:

[0029]

[0030] The Sigmoid function converts the distance into a probability value. The similarity value is between 0 and 1. The larger the value, the more similar it is. By introducing the contrast loss function, the feature space is optimized so that the features from the same person are closer and the features from different people are farther away. The loss function is:

[0031]

[0032] in, Represents the positive sample label, if it is the same person, it is 1, otherwise it is 0, is the boundary distance, is the feature of the i-th extracted input image, For the database The feature vector of the face image, is the sample size.

[0033] As a preferred embodiment of the monitoring method based on face recognition and license plate recognition described in the present invention, the monitoring and analysis based on the recognition results of license plates and faces includes first detecting whether the current driver has a driver's license, and if driving without a license, an alarm is issued. When the identified driver is not the driver corresponding to the license plate, the vehicle type is identified. If the vehicle owner allows a specific driver to drive the vehicle within a specified time period, the system records the identification time and the identified driver.

[0034] When a driver's history record shows that he frequently drives a vehicle that he does not own and does not rent, the system will issue an identity verification prompt.

[0035] When a vehicle enters a restricted area, the system identifies whether there are people in the car who are allowed to enter the restricted area. If not, the system will automatically mark it as suspicious driving, sound an alarm, and record it.

[0036] When the driver matches the car owner, but the non-driving members in the car continuously show faces that have not been recognized in the history and the vehicle is not registered as a passenger vehicle, it is identified as illegal passenger transportation. The illegal passenger transportation identification mechanism is only triggered when the recognized historical faces exceed the preset value.

[0037] Another object of the present invention is to provide a monitoring system based on face recognition and license plate recognition, which can accurately determine whether there is an illegal behavior through multi-source data fusion and real-time calculation, enhance the degree of intelligence, ensure accuracy, and solve the problem that the current driving monitoring method contains a single data source with insufficient monitoring accuracy.

[0038] As a preferred solution of the monitoring system based on face recognition and license plate recognition described in the present invention, it includes:

[0039] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a monitoring method based on face recognition and license plate recognition.

[0040] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a monitoring method based on face recognition and license plate recognition.

[0041] Beneficial effects of the present invention: The monitoring method based on face recognition and license plate recognition provided by the present invention can automatically determine whether the vehicle has problems such as overloading and illegal driving by combining license plate recognition, face recognition and driver behavior analysis, and provide real-time feedback to management personnel through an intelligent early warning system, thereby reducing human intervention and misjudgment. The judgment is made in combination with vehicle type and driver information, and the analysis is more comprehensive, efficiently and accurately assisting management personnel. The present invention achieves better results in terms of automation and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0043] Figure 1 An overall flow chart of a monitoring method based on face recognition and license plate recognition provided for the first embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0045] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a monitoring method based on face recognition and license plate recognition, including: S1: acquiring images and video streams, and performing data processing on the acquired data.

[0046] Furthermore, obtaining images and video streams includes the front-end camera transmitting the video stream to the back-end server through the RTSP protocol, the server decoding the video stream, and extracting the image data of each frame therefrom.

[0047] After the video stream is decoded, the server converts each frame in the video into a static image.

[0048] When there are multiple cameras monitoring the same area, the video stream of each camera will be decoded into independent image frames, marked with timestamps, and each set of image data belongs to the same moment.

[0049] The resolution of the camera's video stream directly affects the quality of the image. Setting it to 1920x1080 (1080p) as a commonly used high-definition resolution can provide enough image details for subsequent processing.

[0050] Frames per second (FPS) controls the smoothness of the video stream. For real-time monitoring, setting it to 30FPS can ensure the real-time nature of image acquisition while ensuring smoothness.

[0051] Use RTSP (Real Time Streaming Protocol) because it is widely used in video surveillance and supports low-latency video transmission.

[0052] It should be noted that the data processing of the acquired data includes processing the image after the image data is acquired. When the image environment is a rainy day, night or cloudy day, histogram equalization is used to improve the brightness contrast of the image.

[0053] Use Gaussian filtering to reduce noise in the image, and the Gaussian filter kernel size is 5x5.

[0054] After removing the noise, the Canny edge detection algorithm is used to extract the obvious edge features in the image. The high threshold is set to 150 and the low threshold is set to 50. Edge pixels below the low threshold are identified as non-edges and ignored, and those between the high threshold and the low threshold are identified as weak edges. Weak edges are retained only when connected to strong edges, otherwise they are discarded. Edge pixels above the high threshold are retained. After completing the edge construction, the area containing the vehicle is preliminarily screened out.

[0055] The target objects are pedestrians and vehicles.

[0056] S2: Perform target object recognition to screen out pedestrians, and recognize license plates and faces.

[0057] Furthermore, target object recognition and pedestrian screening include target object detection using YOLOv5. YOLOv5 scans and identifies the categories of target objects in the image, including vehicles and pedestrians, and marks the location of the vehicles in the image. The location of each target is represented by a bounding box.

[0058] YOLOv5 needs to resize the input image to a fixed size for prediction. Choosing an input size of 640x640 can achieve a good balance between processing speed and accuracy.

[0059] For each detected target, YOLOv5 will output a confidence score, which indicates the reliability of the target detection. If the confidence score is lower than the threshold, the detection result will be ignored. It is set to 0.4, that is, the target detection is considered valid only when the confidence score is greater than or equal to 40%.

[0060] YOLOv5 will detect multiple overlapping bounding boxes and suppress redundant boxes through NMS. Set the IoU (Intersection over Union) threshold to 0.5, that is, when the overlap of multiple boxes exceeds 50%, the system will select the box with the highest confidence as the final detection result.

[0061] When YOLOv5 identifies a vehicle, it further detects whether the vehicle contains a license plate area.

[0062] License plate area detection uses the YOLOv5 model to identify the license plate area , the coordinates are the bounding box coordinates, expressed as:

[0063]

[0064] in, Represents the coordinates of the four corners of the license plate bounding box.

[0065] It should be noted that the recognition of license plates and faces includes generating character images through generative adversarial networks, and images generated by GAN The license plate characters are extracted to obtain the enhanced character area, which is expressed as: in, Represents a license plate image.

[0066] The convolutional neural network with attention mechanism is used to recognize license plate characters, which is expressed as: in, Indicates Character area, is the recognition result of each character, and n is the number of license plate characters.

[0067] It should also be noted that the recognition of license plates and faces also includes first extracting facial features through the convolutional neural network and the Siamese network, which is expressed as

[0068] in, and For the two initial input face images, when only one camera obtains data, two consecutive frames of images are selected for calculation. When there are multiple cameras, a group of images at the same time are selected for calculation. For the face images stored in the database and the extracted input image The features are used to calculate the Euclidean distance and converted into similarity through the Sigmoid function, which is expressed as: The Sigmoid function converts the distance into a probability value. The similarity value is between 0 and 1. The larger the value, the more similar it is. By introducing the contrast loss function, the feature space is optimized so that the features from the same person are closer and the features from different people are farther away. The loss function is:

[0069] in, Represents the positive sample label, if it is the same person, it is 1, otherwise it is 0, is the boundary distance, is the feature of the i-th extracted input image, For the database The feature vector of the face image, is the sample size.

[0070] S3: Combine the license plate and face recognition results for monitoring and analysis.

[0071] Furthermore, monitoring and analysis based on the results of license plate and face recognition include first detecting whether the current driver has a driver's license. If the driver is driving without a license, an alarm is sounded. When the identified driver is not the driver corresponding to the license plate, the vehicle type is identified. If the vehicle owner allows a specific driver to drive the vehicle within a specified time period, the system records the identification time and the identified driver.

[0072] When a driver's history record shows that he frequently drives a vehicle that he does not own and does not rent, the system will issue an identity verification prompt.

[0073] When a vehicle enters a restricted area, the system identifies whether there are people in the car who are allowed to enter the restricted area. If not, the system will automatically mark it as suspicious driving, sound an alarm, and record it.

[0074] When the driver matches the car owner, but the non-driving members in the car continuously show faces that have not been recognized in the history and the vehicle is not registered as a passenger vehicle, it is identified as illegal passenger transportation. The illegal passenger transportation identification mechanism is only triggered when the recognized historical faces exceed the preset value.

[0075] Example 2, an embodiment of the present invention, provides a monitoring method based on face recognition and license plate recognition. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0076] First, a traffic monitoring system in a downtown area was selected for the experiment. The area has multiple front-end cameras, which transmit video streams to the back-end server in real time through the RTSP protocol. The server is responsible for decoding and extracting each frame of image data. In addition, YOLOv5 is used for target object recognition, and the license plate image is enhanced by the generative adversarial network (GAN) for character extraction, and the convolutional neural network (CNN) and Siamese network are combined for face recognition.

[0077] Before the experiment, all front-end cameras were synchronized with the vehicle and driver information in the database, and the vehicle license plate and driver's facial data were stored in the system. In the experiment, 20 different time periods and multiple monitoring scenarios were selected, including normal driving, overloading, night driving, etc. Through tests under different weather conditions (such as sunny days, rainy days, nights, etc.), we evaluated the adaptability and accuracy of the monitoring method in practical applications.

[0078] Table 1 Experimental data comparison table

[0079] Testing time Weather conditions Image clarity (%) Target detection accuracy (%) License plate recognition accuracy (%) Face recognition accuracy (%) Success rate of illegal behavior identification (%) 8:00 sunny 95 98 97 96 92 12:00 sunny 92 96 94 92 90 15:00 cloudy day 89 94 92 89 88 18:00 night 85 91 88 87 80 20:00 rain 80 89 85 84 75 22:00 rain 82 90 87 85 78

[0080] Referring to Table 1, it can be seen that image clarity has a significant impact on the accuracy of target detection, license plate recognition and face recognition. On sunny and cloudy days, the image clarity is high, the target detection accuracy is above 96%, and the license plate and face recognition accuracy are also high, reaching 94%-98% and 92%-96% respectively. However, on rainy days and at night, due to insufficient light or weather reasons, the image clarity is low and the recognition accuracy is relatively reduced. For example, on rainy days and at night, the image clarity is the lowest, the license plate recognition accuracy drops to 85%, and the face recognition accuracy drops to 84%. This shows that the present invention effectively enhances the image quality in a complex environment through the processing methods of histogram equalization, Gaussian filtering and Canny edge detection. Although the image clarity decreases under harsh conditions, the recognition results still maintain a high accuracy.

[0081] In the tests on sunny and cloudy days, the license plate recognition accuracy was 94%-97%, and the face recognition accuracy was 92%-96%, indicating that the present invention combines the dual optimization of the generative adversarial network (GAN) and the attention mechanism CNN, which can provide more accurate character extraction in license plate recognition and ensure the high efficiency of license plate recognition even in different weather conditions. In addition, by combining the Siamese network with face recognition, the accuracy of face recognition is relatively stable, and even in low light conditions (such as at night), the recognition accuracy can still be maintained above 85%, showing its strong robustness.

[0082] By comparing the experimental results under different weather conditions, we can see that the image processing technology of the present invention, such as histogram equalization, Gaussian filtering and Canny edge detection, can effectively improve the image quality in harsh environments such as low light and rainy days, thereby maintaining a high recognition accuracy. Compared with the traditional single license plate or face recognition technology, the present invention integrates the recognition results of license plates and faces, and combines behavior analysis and multi-camera synchronous data processing, so that the system can still have a high accuracy in dynamic and complex environments.

[0083] Embodiment 3, an embodiment of the present invention, provides a monitoring system based on face recognition and license plate recognition, including a data acquisition module, a recognition module, and a monitoring and analysis module.

[0084] The data acquisition module is used to obtain images and video streams and process the acquired data. The recognition module is used to identify the target object and screen out pedestrians, and recognize license plates and faces. The monitoring and analysis module is used to perform monitoring and analysis based on the recognition results of license plates and faces.

[0085] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0086] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0087] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0088] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logical function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A monitoring method based on face recognition and license plate recognition, characterized in that: include: Acquire images and video streams, and perform data processing on the acquired data; Perform target object recognition to screen out pedestrians, and recognize license plates and faces; Combine license plate and face recognition results for monitoring and analysis; The license plate and face recognition includes generating a character image using a generative adversarial network (GAN). The license plate characters are extracted to obtain the enhanced character area, which is expressed as: in, Represents a license plate image; The convolutional neural network with attention mechanism is used to recognize license plate characters, which is expressed as: in, Indicates Character area, is the recognition result of each character, n is the number of license plate characters; The license plate and face recognition also includes first extracting face features by convolutional neural network and Siamese network, which is expressed as in, and For the two initial input face images, when only one camera obtains data, two consecutive frames of images are selected for calculation. When there are multiple cameras, a group of images at the same time are selected for calculation. For the face images stored in the database and the extracted input image The features are used to calculate the Euclidean distance and converted into similarity through the Sigmoid function, which is expressed as: The Sigmoid function converts the distance into a probability value. The similarity value is between 0 and 1. The larger the value, the more similar it is. By introducing the contrast loss function, the feature space is optimized so that the features from the same person are closer and the features from different people are farther away. The loss function is: in, Represents the positive sample label, if it is the same person, it is 1, otherwise it is 0, is the boundary distance, is the feature of the i-th extracted input image, For the database The feature vector of the face image, is the sample size.

2. The monitoring method based on face recognition and license plate recognition as claimed in claim 1, characterized in that: The acquisition of images and video streams includes the front-end camera transmitting the video stream to the back-end server via the RTSP protocol, the server decoding the video stream and extracting the image data of each frame therefrom; After the video stream is decoded, the server converts each frame in the video into a static image; When there are multiple cameras monitoring the same area, the video stream of each camera will be decoded into independent image frames, marked with timestamps, and each set of image data belongs to the same moment.

3. The monitoring method based on face recognition and license plate recognition as claimed in claim 2, characterized in that: The data processing of the acquired data includes processing the image after acquiring the image data, and using histogram equalization to improve the brightness contrast of the image when the image environment is a rainy day, night or cloudy day; Use Gaussian filtering to reduce noise in the image, and the Gaussian filter kernel size is 5x5; After removing the noise, the Canny edge detection algorithm is used to extract the obvious edge features in the image. The high threshold is set to 150 and the low threshold is set to 50. The edge pixels below the low threshold are identified as non-edges and ignored. The pixels between the high threshold and the low threshold are identified as weak edges. Weak edges are only retained when connected to strong edges, otherwise they are discarded. The edge pixels above the high threshold are retained. After completing the edge construction, the area containing the vehicle is preliminarily screened out. The target objects are pedestrians and vehicles.

4. The monitoring method based on face recognition and license plate recognition as claimed in claim 3, characterized in that: The target object recognition and pedestrian screening includes using YOLOv5 to detect the target object. YOLOv5 scans and identifies the categories of the target objects in the image, including vehicles and pedestrians, and marks the position of the vehicle in the image. The position of each target is represented by a bounding box. When YOLOv5 identifies a vehicle, it further detects whether the vehicle contains a license plate area; License plate area detection uses the YOLOv5 model to identify the license plate area , the coordinates are the bounding box coordinates, expressed as: in, Represents the coordinates of the four corners of the license plate bounding box.

5. The monitoring method based on face recognition and license plate recognition as claimed in claim 4, characterized in that: The monitoring and analysis of the combined license plate and face recognition results includes first detecting whether the current driver has a driver's license, and if the driver is driving without a license, an alarm is issued, and when the identified driver is not the driver corresponding to the license plate, the vehicle type is identified, and if the vehicle owner is a vehicle that allows a specific driver to drive within a specified time period, the system records the identification time and the identified driver; When a driver's history shows that he frequently drives a vehicle that he does not own and does not rent, the system will issue an identity verification prompt; When a vehicle enters a restricted area, it will identify whether there are people in the car who are allowed to enter the restricted area. If not, the system will automatically mark it as suspicious driving and issue an alarm to record it; When the driver matches the car owner, but the non-driving members in the car continuously show faces that have not been recognized in the history and the vehicle is not registered as a passenger vehicle, it is identified as illegal passenger transportation. The illegal passenger transportation identification mechanism is only triggered when the recognized historical faces exceed the preset value.

6. A system using the monitoring method based on face recognition and license plate recognition as claimed in any one of claims 1 to 5, characterized in that: Including data acquisition module, identification module, monitoring and analysis module; The data acquisition module is used to acquire images and video streams, and perform data processing on the acquired data; The recognition module is used to identify the target object, filter out pedestrians, and recognize license plates and faces; The monitoring and analysis module is used to perform monitoring and analysis in combination with the recognition results of license plates and faces.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the monitoring method based on face recognition and license plate recognition described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the monitoring method based on face recognition and license plate recognition described in any one of claims 1 to 5 are implemented.

Images