Image detection method, device, equipment and storage medium

Abstract

The application provides an image detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring a video stream, each frame of image in the video stream comprising a license plate number, the license plate number being in a license plate detection frame; according to the frame order in the video stream, sequentially comparing an i-th frame of image in the video stream with candidate images in a tracking list, determining a first candidate image in the tracking list which is consistent with the license plate number of the i-th frame of image, and recording the i-th frame of image as a second candidate image, the license plate number in the i-th frame of image being a target license plate, the tracking list comprising at least one candidate image, i being an integer greater than 1; and then determining a target image corresponding to the target license plate according to the first candidate image and the second candidate image, taking the target image as a new first candidate image corresponding to the target license plate, and outputting the new first candidate image. The technical scheme starts from the detection of each frame of image, and realizes more accurate detection of the license plate image.

Description

Technical Field

[0001] This application relates to the field of detection technology, and in particular to an image detection method, apparatus, device, and storage medium. Background Technology

[0002] With economic development, there is an increasing demand for intelligent traffic patrols, and image detection is the most basic and widely used scenario, playing an important role in toll stations, traffic violations, and other incidents.

[0003] In existing technologies, image detection is mostly based on fixed cameras, fixed shooting angles, and fixed lighting conditions, such as parking lot entrances and exits, traffic electronic checkpoints, and other application scenarios. In addition, image detection uses deep learning models, that is, the video is input into the deep learning model to obtain license plate images.

[0004] However, in practical applications, the accuracy of obtaining license plate images based on deep learning models is not high when the video images captured during vehicle movement may be of unclear quality. Summary of the Invention

[0005] This application provides an image detection method, apparatus, device, and storage medium to address the problem of low accuracy in image detection in the prior art.

[0006] In a first aspect, embodiments of this application provide an image detection method, including:

[0007] Acquire a video stream, wherein at least one frame image in the video stream includes at least one license plate detection box, and the license plate detection box contains a license plate number;

[0008] Based on the frame order in the video stream, the i-th frame image in the video stream is compared with the candidate images in the tracking list in turn to determine the first candidate image in the tracking list that matches the license plate number of the i-th frame image, and the i-th frame image is recorded as the second candidate image. The license plate number in the i-th frame image is the target license plate. The tracking list includes at least one candidate image, and i is an integer greater than 1.

[0009] Based on the first candidate image and the second candidate image, the target image corresponding to the target license plate is determined, and the target image is used as the new first candidate image corresponding to the target license plate;

[0010] Output the new first candidate image.

[0011] In one possible design of the first aspect, determining the target image corresponding to the target license plate based on the first candidate image and the second candidate image includes:

[0012] Obtain the frame diagram information of the license plate detection box of the second candidate image. The frame diagram information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence of the characters corresponding to the characters in the license plate detection box of the second candidate image.

[0013] If the block diagram information meets the preset benchmark conditions, the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image are merged to determine the target image corresponding to the target license plate. The benchmark conditions are used to limit whether the second candidate image is superior to the first candidate image.

[0014] Optionally, the benchmark condition is:

[0015] The pixel area of ​​the license plate detection box of the second candidate image is greater than the product of the pixel area of ​​the license plate detection box of the first candidate image and a preset area adjustment parameter.

[0016] The corner confidence of the license plate detection box in the second candidate image is greater than that in the first candidate image.

[0017] The character length of the license plate detection box in the second candidate image is greater than the character length of the license plate detection box in the first candidate image;

[0018] The average confidence level of the characters in the license plate detection box of the second candidate image is greater than the average confidence level of the characters in the license plate detection box of the first candidate image.

[0019] In this possible design, merging the characters of the license plate detection frame of the second candidate image and the characters of the license plate detection frame of the first candidate image to obtain the target image corresponding to the target license plate includes:

[0020] The confidence scores of the characters at each position in the license plate detection frame of the second candidate image are compared sequentially with the confidence scores of the characters at each position in the license plate detection frame of the first candidate image.

[0021] The characters with higher confidence at each position are used as the final recognized characters;

[0022] Based on the final recognized characters at each position, the characters in the license plate detection box in the second candidate image are replaced to obtain the target image corresponding to the target license plate.

[0023] In another possible design of the first aspect, after acquiring the video stream, the method further includes:

[0024] The first frame image is used as a candidate image, and the license plate number corresponding to the candidate image is determined;

[0025] The candidate image and the corresponding license plate number are stored in the tracking list.

[0026] In another possible design of the first aspect, determining, based on the i-th frame image in the video stream and the tracking list, that there exists a first candidate image in the tracking list that matches the license plate number of the i-th frame image includes:

[0027] For each candidate image in the tracking list, the intersection-union ratio of the license plate detection bounding box of the i-th frame image and the license plate detection bounding box of the candidate image is determined based on the i-th frame image and the candidate image;

[0028] When the intersection-union ratio (IU) of the license plate detection box of the i-th frame image and the license plate detection box of the candidate image is greater than a preset IU threshold, it is determined that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image.

[0029] In this possible design, the method further includes:

[0030] When the intersection-union ratio (IU) of the license plate detection box in the i-th frame image and the license plate detection box in each candidate image is less than or equal to the IU threshold, the characters in the i-th frame image are identified.

[0031] The character is input into a first preset model to obtain the confidence level corresponding to the character. The first preset model is a model used to detect the confidence level of characters in an image.

[0032] If the confidence scores corresponding to the characters are all greater than the preset confidence score threshold, for each candidate image, the similarity between the character and the characters in the first candidate image is determined;

[0033] If the similarity is greater than a preset similarity threshold, the candidate image corresponding to the similarity being greater than the similarity threshold is determined to be the first candidate image that matches the license plate number of the i-th frame image.

[0034] Secondly, embodiments of this application provide an image detection method, including: an acquisition module, a processing module, a determination module, and a display module;

[0035] The acquisition module is used to acquire a video stream, wherein at least one frame image in the video stream includes at least one license plate detection box, and the license plate detection box contains a license plate number;

[0036] The processing module is configured to compare the i-th frame image in the video stream with the candidate images in the tracking list according to the frame order in the video stream, determine the first candidate image in the tracking list that matches the license plate number of the i-th frame image, and record the i-th frame image as the second candidate image, wherein the license plate number in the i-th frame image is the target license plate, and the tracking list includes at least one candidate image, wherein i is an integer greater than 1;

[0037] The determining module is used to determine the target image corresponding to the target license plate based on the first candidate image and the second candidate image, and use the target image as the new first candidate image corresponding to the target license plate;

[0038] The display module is used to output the new first candidate image.

[0039] In one possible design of the second aspect, the determining module is specifically used for:

[0040] Obtain the frame diagram information of the license plate detection box of the second candidate image. The frame diagram information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence of the characters corresponding to the characters in the license plate detection box of the second candidate image.

[0041] If the block diagram information meets the preset benchmark conditions, the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image are merged to determine the target image corresponding to the target license plate. The benchmark conditions are used to limit whether the second candidate image is superior to the first candidate image.

[0042] Optionally, the benchmark condition is:

[0043] The pixel area of ​​the license plate detection box of the second candidate image is greater than the product of the pixel area of ​​the license plate detection box of the first candidate image and a preset area adjustment parameter.

[0044] The corner confidence of the license plate detection box in the second candidate image is greater than that in the first candidate image.

[0045] The character length of the license plate detection box in the second candidate image is greater than the character length of the license plate detection box in the first candidate image;

[0046] The average confidence level of the characters in the license plate detection box of the second candidate image is greater than the average confidence level of the characters in the license plate detection box of the first candidate image.

[0047] In this possible design, the determining module merges the characters of the license plate detection frame of the second candidate image and the characters of the license plate detection frame of the first candidate image to obtain the target image corresponding to the target license plate, specifically for:

[0048] The confidence scores of the characters at each position in the license plate detection frame of the second candidate image are compared sequentially with the confidence scores of the characters at each position in the license plate detection frame of the first candidate image.

[0049] The characters with higher confidence at each position are used as the final recognized characters;

[0050] Based on the final recognized characters at each position, the characters in the license plate detection box in the second candidate image are replaced to obtain the target image corresponding to the target license plate.

[0051] In another possible design of the second aspect, after acquiring the video stream, the processing module is further configured to:

[0052] The first frame image is used as a candidate image, and the license plate number corresponding to the candidate image is determined;

[0053] The candidate image and the corresponding license plate number are stored in the tracking list.

[0054] In another possible design, the processing module is specifically used for:

[0055] For each candidate image in the tracking list, the intersection-union ratio of the license plate detection box of the i-th frame image and the license plate detection box of the first candidate image is determined based on the i-th frame image and the candidate image.

[0056] When the intersection-union ratio (IU) of the license plate detection box of the i-th frame image and the license plate detection box of the candidate image is greater than a preset IU threshold, it is determined that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image.

[0057] In this possible design, the processing module is further configured to:

[0058] When the intersection-union ratio (IU) of the license plate detection box of the i-th frame image and the license plate detection box of each first candidate image is less than or equal to the IU threshold, the characters in the i-th frame image are identified.

[0059] The character is input into a first preset model to obtain the confidence level corresponding to the character. The first preset model is a model used to detect the confidence level of characters in an image.

[0060] If the confidence scores corresponding to the characters are all greater than the preset confidence score threshold, for each candidate image, the similarity between the character and the characters in the candidate image is determined;

[0061] If the similarity is greater than a preset similarity threshold, the candidate image corresponding to the similarity being greater than the similarity threshold is determined to be the first candidate image that matches the license plate number of the i-th frame image.

[0062] Thirdly, embodiments of this application provide a computer device, including: a processor and a memory;

[0063] The memory stores computer-executed instructions;

[0064] The processor executes the computer execution instructions, causing the computer device to perform the image detection method as described in the first aspect and various possible designs above.

[0065] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the image detection method as described in the first aspect and various possible designs above.

[0066] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, is used to implement the image detection method as described in the first aspect and various possible designs above.

[0067] The image detection method, apparatus, device, and storage medium provided in this application acquire a video stream, in which at least one frame image includes at least one license plate detection box containing a license plate number. Then, according to the frame order in the video stream, the i-th frame image in the video stream is sequentially compared with candidate images in a tracking list to determine a first candidate image in the tracking list whose license plate number matches that of the i-th frame image. The i-th frame image is then designated as a second candidate image, and the license plate number in the i-th frame image is the target license plate. The tracking list includes at least one candidate image, where i is an integer greater than 1. Subsequently, based on the first and second candidate images, the target image corresponding to the target license plate is determined, and this target image is used as a new first candidate image corresponding to the target license plate, which is then output. This technical solution, starting from the detection of each frame image, achieves more accurate license plate image detection. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0069] Figure 1 This is a schematic diagram illustrating an application scenario of the image detection method provided in the embodiments of this application;

[0070] Figure 2 A schematic flowchart of an embodiment of the image detection method provided in this application;

[0071] Figure 3 A schematic flowchart of Embodiment 2 of the image detection method provided in this application;

[0072] Figure 4 This is a flowchart illustrating Embodiment 3 of the image detection method provided in this application.

[0073] Figure 5 This is a flowchart illustrating Embodiment 4 of the image detection method provided in this application.

[0074] Figure 6 This is a flowchart illustrating Embodiment 5 of the image detection method provided in this application.

[0075] Figure 7 This is a schematic diagram of the structure of the image detection device provided in the embodiments of this application;

[0076] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0077] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0078] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0079] Before introducing the embodiments of this application, the background technology of this application will be explained first:

[0080] With economic development, there is an increasing demand for intelligent traffic patrols, and license plate recognition is the most basic and widely used application scenario. However, traditional license plate recognition is mostly based on fixed cameras, fixed shooting angles, and fixed lighting conditions, such as parking lot entrances and exits, and traffic electronic checkpoints. Moreover, traditional license plate recognition rarely uses deep learning models. In recent years, with the advancement of deep learning models, license plate recognition based on deep learning models has become the mainstream.

[0081] In intelligent transportation patrol applications, including vehicle-mounted and human-mounted systems, the quality, angle, and lighting conditions of captured license plate images are constantly changing. Therefore, accurately detecting, tracking, and identifying license plates amidst these dynamic changes, and selecting the highest-quality images, becomes a pressing practical problem. In existing technologies, simply improving the accuracy of a particular deep learning model does not necessarily enhance robustness in real-world applications.

[0082] In other words, the existing technology has the following problems:

[0083] 1. During the activity period in the video, multiple images of the same license plate were captured, and there were inconsistencies in character recognition.

[0084] The reasons for this problem may be:

[0085] 1. Because not every frame of a video captured during motion is of high quality, the detection model may not be able to detect the target in every frame. Therefore, it is not easy to ensure continuous tracking, which may result in multiple tracking targets for the same license plate.

[0086] 2. The computing power of the computer is limited, the algorithm does not process frame by frame, and it cannot use very complex and time-consuming tracking algorithms, which will further increase the probability of tracking failure.

[0087] 3. During a certain period of continuous tracking, the overall clarity of all frames of the license plate video is not high, which may lead to errors in license plate number recognition.

[0088] Furthermore, when performing object detection based on deep learning models, there is a certain probability of failure, which is also unavoidable. In field testing, the probability of tracking failure is very high, with a repetition rate of 40%-50%.

[0089] Second, the license plate images captured during the video's activity period were of poor quality, with some even having obscured edges, and there were also instances of character recognition errors.

[0090] The reasons for this problem may be:

[0091] 1. Because a reasonable image quality filtering algorithm was not designed during the entire license plate target tracking period, the quality of the final image released at the end of the tracking was poor.

[0092] Furthermore, the design of image quality models involves many parameters, and the types of image quality defects in actual field application scenarios are numerous, making it difficult to design a reasonable quality assessment model that integrates multiple parameters.

[0093] Based on the problems existing in the above-mentioned prior art Figure 1 This diagram illustrates an application scenario of the image detection method provided in this application, used to solve the aforementioned technical problems. For example... Figure 1 As shown in the diagram, the application scenario includes: computer equipment 11, camera 12, and at least one vehicle 13.

[0094] The computer device 11 can be a computer, laptop, PC, tablet, or other similar device. In some implementations, it can also be a server. Specifically, it provides an image detection platform where, after the video captured by the camera 12 is uploaded, the server processes the video and sends the results to the client.

[0095] Optionally, the camera 12 can be installed in parking lots, highway speed measurement sections, highway entrances and exits, or it can be installed on the dashcams of other vehicles to record video in real time while the vehicle is in motion.

[0096] In one possible implementation, when vehicle A is driving on the road, at least one vehicle 13 is operating in front of vehicle A or in an area that can be captured by camera 12. Camera 12 acquires video data in real time (the video data may record part of the driving situation of vehicle 13, but is not limited to the entire video, and vehicle 13 is always present).

[0097] Furthermore, the computer device 11 continuously acquires the video captured by the camera 12, performs frame processing on the video to obtain continuous frame images, and processes each frame image in turn to obtain an image that meets expectations, in which the license plate of the vehicle 13 can be clearly identified.

[0098] It should be understood that vehicle 13 can be at least one vehicle, that is, at a certain moment in the video, the image captured can be of multiple vehicles, among which vehicle 13 can be various sedans, SUVs, trucks, buses, etc.

[0099] Furthermore, for the computer device 11, an algorithm for real-time license plate tracking and recognition can be implemented by combining a deep learning model for object detection and a license plate recognition model. The object detection model is responsible for detecting license plate targets in video image frames, while the license plate recognition model is responsible for recognizing license plate characters. This is detailed in the following embodiments.

[0100] To address the aforementioned technical problems, the inventor's technical conception process is as follows: When processing video, there may be instances where a segment or the entire video is blurred. If the video can be divided into frames to obtain individual frame images, and then each frame image is compared sequentially and continuously optimized, specifically when processing characters in a license plate image, continuously selecting characters with higher confidence levels, the image containing the license plate number can be more accurately identified. This also avoids problems such as repeatedly capturing the same license plate image and character recognition errors found in existing technologies.

[0101] The following is based on Figure 1 The illustrated application scenario diagrams provide a detailed explanation of the technical solutions of this application through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0102] Figure 2 This is a schematic flowchart of an embodiment of the image detection method provided in this application. Figure 2 As shown, the image detection method may include the following steps:

[0103] Step 21: Obtain the video stream.

[0104] In this video stream, at least one frame image includes at least one license plate detection box containing a license plate number.

[0105] In this step, in order to identify the license plate number, the computer equipment needs to capture images of at least one vehicle using a camera to obtain a video stream.

[0106] Optionally, the camera equipment can be a webcam, camcorder, etc. Depending on the application scenario, the installation location of the camera equipment and the location for acquiring video data will also be different. For example, at a traffic light intersection, the camera equipment can be installed on the pole where the indicator light is located; at the entrance and exit of a shopping mall, the camera equipment can be installed near the entrance and exit gates; when a vehicle is in motion, the camera equipment can be installed at the front or rear of the vehicle to film other vehicles on the road.

[0107] Furthermore, the camera device connects to the computer device via wireless or wired connection and sends the real-time video stream to the computer device. After receiving the video stream, the computer device continuously processes the frames to obtain frame images composed in chronological order.

[0108] Optionally, the computer device has a license plate detection frame annotation function, which annotates the license plate number of each frame of the image to obtain the license plate detection frame. In this embodiment, the license plate detection frame is not limited to a two-dimensional detection frame or a three-dimensional detection frame.

[0109] Optionally, after processing the video stream into multiple frames, license plate detection bounding boxes are sequentially labeled for each frame. In some implementations, the following possibilities may exist. For example, taking a total of 5 frames: the first frame has license plate detection bounding boxes for license plate numbers O and P; the second frame has license plate detection bounding boxes for license plate numbers O and Q; the third frame has license plate detection bounding boxes for license plate numbers I and P; the fourth frame does not have any license plate detection bounding boxes identified; and the fifth frame has license plate detection bounding boxes for license plate numbers O, P, and Q.

[0110] Step 22: Based on the frame order in the video stream, compare the i-th frame image in the video stream with the candidate images in the tracking list in turn, determine the first candidate image in the tracking list that matches the license plate number of the i-th frame image, and record the i-th frame image as the second candidate image.

[0111] Wherein, the license plate number in the i-th frame image is the target license plate, the tracking list includes at least one candidate image, and i is an integer greater than 1.

[0112] In this step, when the computer device acquires multiple frames of images, it processes them frame by frame. The starting point is to continuously compare the multiple frames one by one to determine the images with higher quality, and then process these images.

[0113] Optionally, the frame images are processed sequentially. Taking the i-th frame as an example, the i-th frame image is compared with each candidate image in the tracking list to determine whether there is an image with the same license plate number as the i-th frame image. If there is, the candidate image is recorded as the first candidate image and the i-th frame image is recorded as the second candidate image.

[0114] In one possible implementation, the comparison method can be to perform an intersection-union (IUU) comparison of the areas of the license plate detection boxes in the tracking list and the license plate detection boxes of the i-th frame image. When the IUU meets certain conditions, it is considered that this step can directly determine the image with the same license plate number as the i-th frame image in the tracking list.

[0115] In another possible implementation, when the intersection-union ratio does not meet the condition, it is necessary to compare other information of the license plate detection boxes in the tracking list and the license plate detection boxes in the i-th frame image before a determination can be made.

[0116] It should be understood that the detailed descriptions of the two possible implementations mentioned above are provided by Figure 4 The corresponding implementation examples are given, and will not be repeated here.

[0117] Optionally, the tracking list can store images corresponding to different license plate numbers:

[0118] For example, the first frame image is used as a candidate image, and the license plate number corresponding to the candidate image is determined. The candidate image and the license plate number corresponding to the candidate image are then stored in the tracking list.

[0119] For example, when processing the 7th frame in this step, the tracking list already contains frame images corresponding to 6 different license plate numbers. This means that the license plate information corresponding to the first 6 frames is considered to be different. In other words, the first 6 frames are used as candidate images, and their respective license plate numbers and corresponding images are stored in the tracking list.

[0120] Specifically, for example, the header of the tracking list can be different license plate numbers, and the path corresponding to each license plate number stores the frame image that the computer device believes corresponds to that license plate number, i.e., candidate image.

[0121] Furthermore, when comparing the i-th frame image in the video stream according to the frame order in the video stream, if, for example, the license plate detection box is not identified in the 4th frame image, the 5th frame image is processed directly to continue tracking the vehicle corresponding to the target license plate, thus avoiding inaccurate license plate recognition caused by problems such as missing frames.

[0122] It should be understood that when processing the i-th frame image in the embodiments of this application, there may be multiple license plate detection boxes, that is, license plates corresponding to different vehicles. In the embodiments, one license plate detection box is used as an example. When processing the video stream data frame by frame, the processing process may be parallel, that is, multiple license plate detection boxes are tracked and processed simultaneously and separately.

[0123] Step 23: Based on the first candidate image and the second candidate image, determine the target image corresponding to the target license plate, and use the target image as the new first candidate image corresponding to the target license plate.

[0124] In this step, the second candidate image is the i-th frame image being processed, while the first candidate image is the candidate image corresponding to the license plate number in the license plate detection box of the i-th frame image in the tracking list. The information in the first candidate image and the second candidate image is compared and optimized to determine a more accurate image (target image) for the license plate number (target license plate).

[0125] In one possible implementation, this step may involve comparing the information of the license plate detection box in the second candidate image with the information of the license plate detection box in the first candidate image. When certain conditions are met, the characters representing the license plate number in the two license plate detection boxes are merged to determine the target image.

[0126] Optionally, taking a license plate number as an example, after processing in step 22, the set of frame images corresponding to the license plate number can be determined. Each frame image in the set contains the coordinate information of the license plate detection box corresponding to the license plate number. By sequentially determining the coordinates of the license plate detection box corresponding to the license plate number, the trajectory of the vehicle corresponding to the license plate number can be obtained. Alternatively, the coordinates of the license plate detection box in the target image corresponding to the license plate number determined last can also be used.

[0127] Furthermore, since steps 22 and 23 are cyclical processes, after determining the target image corresponding to the target license plate, the target image can be used as the new first candidate image (that is, the original first candidate image corresponding to the target number is deleted), and then the relevant processing of the i+1 frame image is performed.

[0128] Step 24: Output the new first candidate image.

[0129] Optionally, a tracking timeout timer Fn can be set here. When the tracking time Fn of the target license plate is large, the next step can be executed. The judgment of large time can be set with a time threshold T1. When Fn is greater than T1, the target is considered lost.

[0130] In one possible implementation, after processing the frame images in the video stream sequentially for a certain period of time, when it is determined that there is no detection box corresponding to the target license plate, it is considered that the vehicle corresponding to the target license plate has left the area that the camera device can capture. At this time, the latest first candidate image corresponding to the target license plate in the tracking list can be output.

[0131] Furthermore, the new first candidate image is an optimized image, in which the characters in the license plate detection box have high recognizability and accuracy. Before outputting the target image to the display screen of a computer device or a terminal connected to the computer device, a filtering operation can be performed to obtain an image that is more satisfactory to the user.

[0132] Optionally, to further improve user satisfaction, the target image can be filtered before displaying it. For example, if a target image is detected to have pixel A and a pixel threshold of B, the target image can be output when A is greater than B.

[0133] It should be understood that the pixel threshold as a filtering condition is only an example, and other filtering indicators, such as sharpness, can also be used.

[0134] In one possible implementation, since the characters corresponding to the target license plate in the new first candidate image have been determined in the above steps, not only can the new first candidate image be output, but also the characters corresponding to the target license plate can be output.

[0135] The image detection method provided in this application acquires a video stream, in which at least one frame image includes at least one license plate detection box containing a license plate number. Then, according to the frame order in the video stream, it sequentially compares the i-th frame image with candidate images in a tracking list to determine a first candidate image in the tracking list whose license plate number matches that of the i-th frame image. This i-th frame image is then designated as a second candidate image, and the license plate number in the i-th frame image is the target license plate. The tracking list includes at least one candidate image, where i is an integer greater than 1. Based on the first and second candidate images, the target image corresponding to the target license plate is determined, and this target image is used as a new first candidate image corresponding to the target license plate, which is then output. This technical solution, starting from the detection of each frame image, achieves more accurate license plate image detection.

[0136] Based on the above embodiments, Figure 3 This is a schematic flowchart of Embodiment 2 of the image detection method provided in this application. Figure 3 As shown, step 23 in the above embodiment may include the following steps:

[0137] Step 31: Obtain the bounding box information of the license plate detection box of the second candidate image.

[0138] The block diagram information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence of the characters corresponding to the characters in the license plate detection box of the second candidate image.

[0139] In this step, the license plate detection bounding box of the second candidate image is processed to obtain the pixel area Zi of the license plate detection bounding box, the corner confidence Pi, the character length Ln, and the average confidence An of the corresponding character.

[0140] The corner points of the license plate detection frame can be the four corner points of the license plate detection frame, i.e., the key points; the character length of the license plate detection frame can be the number of characters in the license plate number.

[0141] As an example, the license plate number 31290M has a character length of 6. The average confidence level of the characters in the license plate detection frame can be 95%. Specifically, the confidence levels of the license plate number 31290M are 98%, 92%, 97%, 93%, 96%, and 94%, respectively. The average value is:

[0142] (98% + 92% + 97% + 93% + 96% + 94%) / 6 = 95%

[0143] Step 32: If the block diagram information meets the preset benchmark conditions, merge the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image to determine the target image corresponding to the target license plate.

[0144] In this step, based on the bounding box information of the license plate detection box of the second candidate image and the bounding box information of the license plate detection box of the first candidate image, when determining whether the second candidate image is better than the first candidate image, the bounding box information of the two license plate detection boxes is merged, that is, the character with higher confidence is used as the character of the target image.

[0145] In other words, the benchmark condition is used to determine whether the second candidate image is superior to the first candidate image. Specifically, the benchmark condition may include the following:

[0146] The pixel area Zi of the license plate detection box in the first and second candidate images is greater than the product Zo of the pixel area of ​​the license plate detection box in the first candidate image and the preset area adjustment parameter.

[0147] The preset area adjustment parameter can be 'a', with a value range of 0-1.

[0148] The corner confidence Pi of the license plate detection box in the second candidate image is greater than the corner confidence Po of the license plate detection box in the first candidate image;

[0149] Third, the character length Ln of the license plate detection box in the second candidate image is greater than the character length Lo of the license plate detection box in the first candidate image;

[0150] Fourth, the average confidence level An of the characters in the license plate detection boxes of the second candidate image is greater than the average confidence level Ao of the characters in the license plate detection boxes of the first candidate image.

[0151] Furthermore, based on the bounding box information of the license plate detection box in the second candidate image and the bounding box information of the license plate detection box in the first candidate image, the above conditions are judged. When the license plate detection boxes of the second candidate image and the first candidate image meet the baseline conditions, the character merging operation of the license plate detection boxes is performed.

[0152] It should be understood that the number and content of these benchmark conditions can be increased or decreased, and are not limited to the four conditions mentioned above.

[0153] Optionally, one possible implementation of the character merging operation in this step may specifically include the following steps:

[0154] Step 1: Compare the confidence scores of the characters at each position in the license plate detection box of the second candidate image with the confidence scores of the characters at each position in the license plate detection box of the first candidate image.

[0155] Optionally, by detecting each character in the license plate detection frame, the confidence level of each character at its position in the license plate detection frame of the second candidate image and the first candidate image can be obtained respectively.

[0156] For example, the confidence levels of each character in the license plate detection box in the second candidate image are 98%, 97%, 93%, 98%, 99%, and 96%, respectively, while the confidence levels of each character in the license plate detection box in the first candidate image are 97%, 96%, 95%, 93%, and 99%, respectively. Comparing the confidence levels at each position sequentially, we find that: for the first position, 98% is greater than 97%; for the second position, 97% is greater than 96%; for the third position, 98% is greater than 95%; for the fourth position, 99% is greater than 93%; and for the fifth position, 96% is less than 99%.

[0157] Step 2: Select the characters with higher confidence levels at each position as the final characters to be recognized;

[0158] Optionally, the character with the higher confidence level in the corresponding position of the two license plate detection frames at each of the above positions can be selected as the desired character.

[0159] For example, the first character in the second candidate image (98% greater than 97% in the first position), the second character in the second candidate image (97% greater than 96% in the second position), the third character in the second candidate image (98% greater than 95% in the third position), the fourth character in the second candidate image (99% greater than 93% in the fourth position), and the fifth character in the first candidate image (96% less than 99% in the fifth position) are used as the final recognized characters.

[0160] Optionally, for example, if the confidence levels of characters at a certain position are the same, the character with the higher confidence level can be considered as any one of the characters at that position.

[0161] Step 3: Replace the characters in the license plate detection box in the second candidate image with the final recognized characters at each position to obtain the target image corresponding to the target license plate.

[0162] Optionally, since the second candidate image is better than the first candidate image, the target image corresponding to the target license plate can be obtained by replacing the characters in the license plate detection box in the second candidate image with the final recognized characters, based on the second candidate image.

[0163] For example, as mentioned above, the first, second, third, and fourth characters in the second candidate image remain unchanged, while the fifth character in the second candidate image is replaced with the fifth character in the first candidate image.

[0164] The image detection method provided in this application obtains the frame information of the license plate detection box of the second candidate image. Then, when the frame information meets the preset benchmark conditions, the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image are merged to determine the target image corresponding to the target license plate. This technical solution compares the information of the license plate detection boxes of the second candidate image and the first candidate image to obtain a more accurate target image corresponding to the target license plate.

[0165] Based on the above embodiments, Figure 4 This is a schematic flowchart of Embodiment 3 of the image detection method provided in this application. Figure 4 As shown, the implementation of step 22 in the above embodiments may include the following steps:

[0166] Step 41: For each candidate image in the tracking list, determine the intersection-union ratio of the license plate detection box of the i-th frame image and the license plate detection box of the candidate image based on the i-th frame image and the candidate image.

[0167] In this step, it is necessary to determine which license plate number in the tracking list matches the license plate number in the i-th frame image, so that the images with the same number can be processed in step 22. At this time, the intersection-union ratio (IUU) of the license plate detection box of the i-th frame image and the license plate detection boxes of each candidate image in the tracking list can be determined by comparing the license plate detection box of the i-th frame image with the license plate detection boxes of each candidate image.

[0168] Since the intersection-over-union ratio (IoU) describes the degree of overlap between two license plate detection boxes, it can also be seen as a set of two pixels. The IoU of a license plate detection box is equal to the area of ​​the overlapping part of the two license plate detection boxes divided by the area of ​​the merged license plate detection boxes.

[0169] For example, if the license plate detection box of the i-th frame image is E, and the license plate detection box of a candidate image in the tracking list is G, then the intersection-union ratio Hi can be (E∩G) / (E∪G).

[0170] Step 42: When the intersection-union ratio (IU) of the license plate detection box of the i-th frame image and the license plate detection box of the candidate image is greater than the preset IU threshold, it is determined that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image.

[0171] In this step, the intersection-union ratio (IU / U) of the license plate detection box of the i-th frame image and the license plate detection boxes of each candidate image is determined sequentially. The license plate detection box of the candidate image corresponding to the maximum IU / U is found, and the maximum value is judged. When a certain condition is met, it is considered that the license plate number indicated by the license plate detection box of the candidate image is consistent with the license plate number indicated by the license plate detection box of the i-th frame image.

[0172] In one possible implementation, the condition could be a preset cross-union ratio (CUNR) threshold J. When the maximum CUNR Himax is greater than the CUNR threshold J, it is considered that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image.

[0173] Optionally, when the intersection-union ratio (IU) of the license plate detection bounding box of the i-th frame image and the license plate detection bounding boxes of each candidate image is less than or equal to the IU threshold, this step can be implemented in another possible way to determine that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image, specifically:

[0174] Step 1: Identify the characters in the i-th frame of the image.

[0175] Optionally, for the license plate detection box in the i-th frame image, the characters in the license plate detection box are recognized to obtain the characters corresponding to the license plate number.

[0176] Step 2: Input the character into the first preset model to obtain the confidence level of the character. The first preset model is a model used to detect the confidence level of characters in an image.

[0177] Optionally, the characters are sequentially input into a model used to detect the confidence of characters in an image to obtain the confidence of each character.

[0178] For example, taking a character count of 5, the confidence levels for each character are 80%, 90%, 85%, 96%, and 88%.

[0179] It should be understood that, in the above Figure 3 In this context, the method for obtaining the confidence level corresponding to a character can also be as shown in this step.

[0180] Step 3: If the confidence scores corresponding to all characters are greater than the preset confidence threshold, determine the similarity between the characters in each candidate image and the characters in the first candidate image.

[0181] Optionally, after obtaining the confidence levels of each character as 80%, 90%, 85%, 96%, and 88%, it is determined whether the confidence level of each character is greater than the preset confidence threshold at each position.

[0182] For example, if the confidence thresholds for each position of the five characters are preset to be 78%, 88%, 81%, 95%, and 80% (the confidence thresholds for each position can also be the same), then the confidence of each character can be obtained as greater than the confidence threshold for its respective position.

[0183] It should be understood that if there is at least one non-compliant size relationship, the license plate detection box of the i-th frame image is discarded, and the processing of the next frame image is performed.

[0184] Furthermore, similarity can be determined by judging the similarity between the characters in the i-th frame image and the characters in each candidate image.

[0185] For example, if the license plate characters in the i-th frame are ABD123 and the license plate characters in a candidate image are BC133, and the common part B-1-3 is found in both license plates, then the similarity is recorded as 3. Similarly, if the license plate characters in the (i+1)-th frame are ABD123 and the license plate characters in a candidate image are CBC126, and the common part B-12 is found in both license plates, then the similarity is recorded as 3.

[0186] Step 4: If the similarity is greater than the preset similarity threshold, determine the candidate image corresponding to the similarity greater than the similarity threshold as the first candidate image that matches the license plate number of the i-th frame image.

[0187] Optionally, candidate images with a similarity greater than a preset similarity threshold are identified. For example, if the similarity between a certain image and the characters in the i-th frame image is 5, and the similarity threshold is 4, then the candidate image is considered to be consistent with the license plate number in the i-th frame image and is recorded as the first candidate image.

[0188] That is, the first candidate image that matches the license plate number of the i-th frame image was identified in the tracking list.

[0189] The image detection method provided in this application determines the intersection-union ratio (IU / R) of the license plate detection boxes of the i-th frame and the candidate image for each first candidate image in the tracking list, based on the i-th frame image and the first candidate image. If the IU / R of the license plate detection boxes of the i-th frame and the candidate image is greater than a preset IU / R threshold, it is determined that a first candidate image with the same license plate number as the i-th frame image exists in the tracking list. This technical solution, starting from the IU / R relationship of the detection boxes between frame images, identifies frame images with the same license plate number, providing a basis for subsequently obtaining the target image of that number.

[0190] Based on the above embodiments, Figure 5 This is a schematic flowchart of Embodiment 4 of the image detection method provided in this application. Figure 5 As shown, a macroscopic overview of the embodiments of this application is provided:

[0191] Step 1, Begin;

[0192] Step 2: Calculate the intersection-union ratio (IUU) between the current license plate detection bounding box and the candidate images in the tracking list;

[0193] Step 3: Select the largest intersection-union ratio;

[0194] Step 4: Determine if the maximum crossover-union ratio (CUI) is greater than the CUI threshold. If yes, proceed to step 10; otherwise, proceed to step 5.

[0195] Step 5: Identify the confidence level of the characters in the current license plate detection frame;

[0196] Step 6: Determine if the confidence level of the character is greater than the confidence threshold. If yes, proceed to step 8; otherwise, proceed to step 7.

[0197] Step 7: Discard the current license plate detection frame;

[0198] Step 8: Determine the similarity between the characters in the current license plate detection box and the characters in the candidate image with the highest intersection-union ratio;

[0199] Step 9: Determine if the similarity is greater than the similarity threshold. If yes, proceed to step 10; otherwise, proceed to step 14.

[0200] Step 10: Refresh tracking parameters (tracking detection box coordinates pos and tracking timeout timer Fn);

[0201] Step 11: Select the best image;

[0202] Step 12: Determine if the time for tracking the license plate detection frame coordinates is greater than the time threshold. If yes, proceed to step 13; otherwise, proceed to step 15.

[0203] Step 13: Image filtering (i.e., outputting images where the number of pixels in the target image is greater than the preset pixel threshold), then proceed to step 15;

[0204] Step 14: Obtain the next frame image, then proceed to step 10;

[0205] Step 15, End.

[0206] The image detection method provided in this application calculates the intersection-union ratio (IUGR) of the current license plate detection frame and candidate images in the tracking list, selects the largest IUGR, and when the largest IUGR is greater than an IUGR threshold, performs image optimization based on the candidate image corresponding to the largest IUGR and the current license plate detection frame. After meeting a time constraint, the image with more than a preset pixel threshold in the target image is output. When the largest IUGR is not greater than the IUGR threshold, the character information in the current license plate detection frame and the candidate image corresponding to the largest IUGR is used to determine if the current license plate detection frame and the candidate image belong to the same license plate number. Subsequent operations such as image optimization are then performed. This technical solution starts from the candidate image corresponding to the largest IUGR and the current license plate detection frame, enabling the processing of information from different images of the same license plate number, resulting in a more accurate output of images that meet user requirements.

[0207] Furthermore, Figure 6 This is a schematic flowchart of Embodiment 5 of the image detection method provided in this application. Figure 6As shown, for the above Figure 5 Step 11 will be described in detail:

[0208] Step 1, Begin;

[0209] Step 2: Obtain the first area of ​​the current license plate detection box and the second area of ​​the license plate detection box in the candidate image corresponding to the largest intersection-union ratio;

[0210] Step 3: Determine if the first area is greater than a times the second area. If yes, proceed to step 4; otherwise, proceed to step 12.

[0211] Step 4: Obtain the confidence score of the first corner point of the current license plate detection box and the confidence score of the second corner point of the license plate detection box in the candidate image corresponding to the largest intersection-union ratio;

[0212] Step 5: Determine if the confidence level of the first corner point is greater than the confidence level of the second corner point. If yes, proceed to step 6; otherwise, proceed to step 12.

[0213] Step 6: Obtain the length of the first character of the current license plate detection box and the length of the second character of the license plate detection box in the candidate image corresponding to the largest intersection-union ratio;

[0214] Step 7: Determine if the length of the first character is greater than the length of the second character. If yes, proceed to step 8; otherwise, proceed to step 12.

[0215] Step 8: Obtain the first average confidence score of the characters in the current license plate detection box and the second average confidence score of the characters in the license plate detection box in the candidate image with the largest intersection-union ratio;

[0216] Step 9: Determine if the first average is greater than the second average. If yes, proceed to step 10; otherwise, proceed to step 12.

[0217] Step 10: Merge the characters in the current license plate detection box with the characters in the license plate detection box in the candidate image with the highest intersection-union ratio;

[0218] Step 11: Determine the target image;

[0219] Step 12, End.

[0220] The image detection method provided in this application embodiment obtains the first area of ​​the current license plate detection box and the second area of ​​the license plate detection box in the candidate image corresponding to the largest intersection-union ratio (IU). It also obtains the first corner confidence of the current license plate detection box and the second corner confidence of the license plate detection box in the candidate image corresponding to the largest IU, the first character length of the current license plate detection box and the second character length of the license plate detection box in the candidate image corresponding to the largest IU. Furthermore, it obtains the first average confidence of the characters in the current license plate detection box and the second average confidence of the characters in the candidate image corresponding to the largest IU. In comparing the information obtained from these four items, if the information of the current license plate detection box is superior to that of the license plate detection box in the candidate image corresponding to the largest IU, then the characters of the current license plate detection box and the characters of the license plate detection box in the candidate image corresponding to the largest IU are merged, and based on this, the target image is determined. This technical solution continuously compares and optimizes frame images, providing a foundation for outputting a license plate image that meets the expected requirements.

[0221] Based on the above method embodiments, Figure 7 This is a schematic diagram of the structure of an image detection device provided in an embodiment of this application. It includes: an acquisition module 71, a processing module 72, a determination module 73, and a display module 74.

[0222] The acquisition module 71 is used to acquire a video stream, wherein at least one frame image in the video stream includes at least one license plate detection box, and the license plate detection box contains a license plate number;

[0223] Processing module 72 is used to compare the i-th frame image in the video stream with the candidate images in the tracking list according to the frame order in the video stream, determine the first candidate image in the tracking list that matches the license plate number of the i-th frame image, and record the i-th frame image as the second candidate image. The license plate number in the i-th frame image is the target license plate. The tracking list includes at least one candidate image, and i is an integer greater than 1.

[0224] The determining module 73 is used to determine the target image corresponding to the target license plate based on the first candidate image and the second candidate image, and use the target image as the new first candidate image corresponding to the target license plate;

[0225] Display module 74 is used to output a new first candidate image.

[0226] In one possible design of this application embodiment, the determining module 73 is specifically used for:

[0227] Obtain the bounding box information of the license plate detection box of the second candidate image. The bounding box information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence of the characters corresponding to the characters in the license plate detection box of the second candidate image.

[0228] If the block diagram information meets the preset benchmark conditions, the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image are merged to determine the target image corresponding to the target license plate. The benchmark conditions are used to limit whether the second candidate image is better than the first candidate image.

[0229] Optional, the baseline condition is:

[0230] The pixel area of ​​the license plate detection box in the second candidate image is greater than the product of the pixel area of ​​the license plate detection box in the first candidate image and the preset area adjustment parameter.

[0231] The corner confidence of the license plate detection box in the second candidate image is greater than that in the first candidate image.

[0232] The character length of the license plate detection box in the second candidate image is greater than the character length of the license plate detection box in the first candidate image;

[0233] The average confidence score of the characters in the license plate detection box of the second candidate image is greater than the average confidence score of the characters in the license plate detection box of the first candidate image.

[0234] In this possible design, the determining module 73 merges the characters of the license plate detection box in the second candidate image and the characters of the license plate detection box in the first candidate image to obtain the target image corresponding to the target license plate, specifically for:

[0235] The confidence scores of the characters at each position in the license plate detection box of the second candidate image are compared with the confidence scores of the characters at each position in the license plate detection box of the first candidate image.

[0236] The characters with higher confidence at each position are used as the final recognized characters;

[0237] Based on the final recognized characters at each position, replace the characters in the license plate detection box in the second candidate image to obtain the target image corresponding to the target license plate.

[0238] In another possible design of this application embodiment, after acquiring the video stream, the processing module 72 is further configured to:

[0239] The first frame image is used as the first candidate image, and the license plate number corresponding to the first candidate image is determined.

[0240] Store the first candidate image and the corresponding license plate number in the tracking list.

[0241] In another possible design of this application embodiment, the processing module 72 is specifically used for:

[0242] For each first candidate image in the tracking list, the intersection-union ratio of the license plate detection box of the i-th frame image and the license plate detection box of the first candidate image is determined based on the i-th frame image and the first candidate image;

[0243] If the intersection-union ratio (IU / U) of the license plate detection bounding box of the i-th frame image and the license plate detection bounding box of the first candidate image is greater than a preset IU / U threshold, it is determined that there is a first candidate image in the tracking list that matches the license plate number of the i-th frame image.

[0244] In this possible design, the processing module 72 is also used for:

[0245] If the intersection-union ratio (IU) of the license plate detection box in the i-th frame image and the license plate detection box in each first candidate image is less than or equal to the IU threshold, then the characters in the i-th frame image are identified.

[0246] The character is input into the first preset model to obtain the confidence level of the character. The first preset model is a model used to detect the confidence level of characters in an image.

[0247] If the confidence scores corresponding to all characters are greater than the preset confidence threshold, for each first candidate image, the similarity between the character and the characters in the first candidate image is determined;

[0248] If the similarity is greater than the preset similarity threshold, the first candidate image corresponding to the similarity greater than the similarity threshold is determined to have the same license plate number as the i-th frame image.

[0249] The image detection device provided in this application embodiment can be used to execute the technical solution corresponding to the image detection method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0250] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented through processing element calls in software, while others are implemented in hardware. Moreover, these modules can be fully or partially integrated together, or implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0251] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of this application. For example... Figure 8 As shown, the computer device may include: a processor 80, a memory 81, and computer program instructions stored in the memory 81 and executable on the processor 80.

[0252] The computer device can be a computer, tablet, or other device with a display function.

[0253] The processor 80 executes computer execution instructions stored in the memory 81, causing the processor 80 to perform the scheme in the above embodiments. The processor 80 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0254] Optionally, the computer device may also include a transceiver 82.

[0255] The memory 81 and transceiver 82 are connected to the processor 80 via the system bus and communicate with each other. The memory 81 is used to store computer program instructions.

[0256] The transceiver 82 is used to communicate with other devices, and the transceiver 82 constitutes a communication interface.

[0257] Optionally, in terms of hardware implementation, the above Figure 7 The acquisition module 71 in the illustrated embodiment corresponds to the transceiver 82 in this embodiment.

[0258] In one possible implementation, the computer device may further include a display for displaying the computer device's display interface, specifically, as in the example of the embodiment, a target image. In hardware implementation, this corresponds to the above... Figure 7The display module 74 in the illustrated embodiment.

[0259] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0260] The computer device provided in this application embodiment can be used to execute the technical solution corresponding to the image detection method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0261] This application also provides a chip for executing instructions, which is used to execute the image detection method described in the above embodiments.

[0262] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer device, the computer device performs the technical solution of the image detection method described in the above embodiments.

[0263] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to perform the technical solution of the image detection method described in the above embodiments.

[0264] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible by general-purpose or special-purpose computer equipment.

[0265] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image detection method, characterized in that, include: Acquire a video stream, wherein at least one frame image in the video stream includes at least one license plate detection box, and the license plate detection box contains a license plate number; Based on the frame order in the video stream, compare the frames sequentially in the video stream. Frame images and candidate images in the tracking list are used to determine the images in the tracking list that are related to the first frame image. The first candidate image that matches the license plate number of the frame image, and the first... The frame image is denoted as the second candidate image, and the first... The license plate number in the frame image is the target license plate, and the tracking list includes at least one candidate image. It is an integer greater than 1; Obtain the bounding box information of the license plate detection box of the second candidate image. The bounding box information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence score of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence score of the characters corresponding to the characters in the license plate detection box of the second candidate image. If the bounding box information meets a preset benchmark condition, merge the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image to determine the target image corresponding to the target license plate. The benchmark condition is used to limit whether the second candidate image is superior to the first candidate image. Use the target image as the new first candidate image corresponding to the target license plate. Output the new first candidate image.

2. The method according to claim 1, characterized in that, The baseline conditions are: The pixel area of ​​the license plate detection box of the second candidate image is greater than the product of the pixel area of ​​the license plate detection box of the first candidate image and a preset area adjustment parameter. The corner confidence of the license plate detection box in the second candidate image is greater than that in the first candidate image. The character length of the license plate detection box in the second candidate image is greater than the character length of the license plate detection box in the first candidate image; The average confidence level of the characters in the license plate detection box of the second candidate image is greater than the average confidence level of the characters in the license plate detection box of the first candidate image.

3. The method according to claim 1 or 2, characterized in that, The step of merging the characters in the license plate detection frame of the second candidate image and the characters in the license plate detection frame of the first candidate image to determine the target image corresponding to the target license plate includes: The confidence scores of the characters at each position in the license plate detection frame of the second candidate image are compared sequentially with the confidence scores of the characters at each position in the license plate detection frame of the first candidate image. The characters with higher confidence at each position are used as the final recognized characters; Based on the final recognized characters at each position, the characters in the license plate detection box in the second candidate image are replaced to obtain the target image corresponding to the target license plate.

4. The method according to claim 1, characterized in that, After acquiring the video stream, the method further includes: The first frame image is used as a candidate image, and the license plate number corresponding to the candidate image is determined; The candidate image and the corresponding license plate number are stored in the tracking list.

5. The method according to claim 1 or 4, characterized in that, The step involves determining, based on the i-th frame image in the video stream and the tracking list, that there exists a match between the i-th frame and the tracking list. The first candidate images with license plate numbers matching the frame images include: For each candidate image in the tracking list, according to the first The frame image and the candidate image are used to determine the first... The intersection-over-union ratio (IoU) of the license plate detection bounding boxes in the frame image and the license plate detection bounding boxes in the candidate image; When the first If the intersection-union ratio (IoU) of the license plate detection bounding box in the frame image and the license plate detection bounding box in the candidate image is greater than a preset IoU threshold, it is determined that there exists a license plate detection bounding box in the tracking list that matches the first... The first candidate image that matches the license plate number of the frame image.

6. The method according to claim 5, characterized in that, The method further includes: When the first If the intersection-union ratio (IoU) of the license plate detection bounding box in the frame image and the license plate detection bounding box in each candidate image is less than or equal to the IoU threshold, then the license plate detection bounding box in the first frame image is identified. Characters in a frame image; The character is input into a first preset model to obtain the confidence level corresponding to the character. The first preset model is a model used to detect the confidence level of characters in an image. If the confidence scores corresponding to the characters are all greater than the preset confidence score threshold, for each candidate image, the similarity between the character and the characters in the candidate image is determined; If the similarity is greater than a preset similarity threshold, the candidate image corresponding to the similarity greater than the similarity threshold is determined to be similar to the first... The first candidate image that matches the license plate number of the frame image.

7. An image detection device, characterized in that, include: The module includes an acquisition module, a processing module, a determination module, and a display module. The acquisition module is used to acquire a video stream, wherein at least one frame image in the video stream includes at least one license plate detection box, and the license plate detection box contains a license plate number; The processing module is configured to compare the frames in the video stream sequentially according to the frame order. Frame images and candidate images in the tracking list are used to determine the images in the tracking list that are related to the first frame image. The first candidate image that matches the license plate number of the frame image, and the first... The frame image is denoted as the second candidate image, and the first... The license plate number in the frame image is the target license plate, and the tracking list includes at least one candidate image. It is an integer greater than 1; The determining module is used to obtain the frame diagram information of the license plate detection box of the second candidate image. The frame diagram information includes: the pixel area of ​​the license plate detection box of the second candidate image, the corner confidence of the license plate detection box of the second candidate image, the character length of the license plate detection box of the second candidate image, and the average confidence of the characters corresponding to the characters of the license plate detection box of the second candidate image. If the frame diagram information meets the preset benchmark conditions, the characters of the license plate detection box of the second candidate image and the characters of the license plate detection box of the first candidate image are merged to determine the target image corresponding to the target license plate. The benchmark conditions are used to limit whether the second candidate image is superior to the first candidate image. The target image is used as the new first candidate image corresponding to the target license plate. The display module is used to output the new first candidate image.

8. The apparatus according to claim 7, characterized in that, The baseline conditions are: The pixel area of ​​the license plate detection box of the second candidate image is greater than the product of the pixel area of ​​the license plate detection box of the first candidate image and a preset area adjustment parameter. The corner confidence of the license plate detection box in the second candidate image is greater than that in the first candidate image. The character length of the license plate detection box in the second candidate image is greater than the character length of the license plate detection box in the first candidate image; The average confidence level of the characters in the license plate detection box of the second candidate image is greater than the average confidence level of the characters in the license plate detection box of the first candidate image.

9. The apparatus according to claim 7, characterized in that, The determining module merges the characters in the license plate detection frame of the second candidate image and the characters in the license plate detection frame of the first candidate image to determine the target image corresponding to the target license plate, specifically for: The confidence scores of the characters at each position in the license plate detection frame of the second candidate image are compared sequentially with the confidence scores of the characters at each position in the license plate detection frame of the first candidate image. The characters with higher confidence at each position are used as the final recognized characters; Based on the final recognized characters at each position, the characters in the license plate detection box in the second candidate image are replaced to obtain the target image corresponding to the target license plate.

10. The apparatus according to claim 9, characterized in that, After acquiring the video stream, the processing module is further configured to: The first frame image is used as a candidate image, and the license plate number corresponding to the candidate image is determined; The candidate image and the corresponding license plate number are stored in the tracking list.

11. The apparatus according to claim 7 or 10, characterized in that, The processing module is specifically used for: For each candidate image in the tracking list, according to the first The frame image and the candidate image are used to determine the first... The intersection-over-union ratio (IoU) of the license plate detection bounding boxes in the frame image and the license plate detection bounding boxes in the candidate image; When the first If the intersection-union ratio (IoU) of the license plate detection bounding box in the frame image and the license plate detection bounding box in the candidate image is greater than a preset IoU threshold, it is determined that there exists a license plate detection bounding box in the tracking list that matches the first... The first candidate image that matches the license plate number of the frame image.

12. The apparatus according to claim 11, characterized in that, The processing module is further configured to: When the first If the intersection-union ratio (IoU) of the license plate detection bounding box in the frame image and the license plate detection bounding box in each candidate image is less than or equal to the IoU threshold, then the license plate detection bounding box in the first frame image is identified. Characters in a frame image; The character is input into a first preset model to obtain the confidence level corresponding to the character. The first preset model is a model used to detect the confidence level of characters in an image. If the confidence scores corresponding to the characters are all greater than the preset confidence score threshold, for each candidate image, the similarity between the character and the characters in the candidate image is determined; If the similarity is greater than a preset similarity threshold, the candidate image corresponding to the similarity greater than the similarity threshold is determined to be similar to the first... The first candidate image that matches the license plate number of the frame image.

13. A computer device, comprising: A processor, a memory, and computer program instructions stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program instructions, implements the image detection method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image detection method as described in any one of claims 1 to 6.

Images