Image processing method, device and computer readable storage medium
By identifying the vehicle image type and applying specific image processing rules, the automatic identification of vin codes and license plates is achieved, and the problem of low manual entry efficiency in the prior art is solved, and the recognition efficiency and accuracy are improved.
Patent Information
- Application Number
- CN201910015377.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-01-04
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-01-04
AI Technical Summary
In the prior art, the identification of vehicle vin codes or license plates requires manual entry, which is inefficient and difficult to achieve automated identification.
By acquiring the vehicle image, the image type is determined to be a vin code image or a license plate image using preset recognition rules, and the corresponding processing rules are used to perform top hat transformation, edge detection, and morphological operations to identify the vin code or license plate area.
It improves the efficiency of vin code or license plate recognition of vehicle images, improves the flexibility and accuracy of image processing, and reduces manual intervention.
Smart Images

Figure CN109740547B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and computer-readable storage medium. Background Art
[0002] At present, we often encounter scenarios where we need to obtain vehicle license plate or vehicle identification number (VIN) information, such as when handling vehicle business. In some business scenarios, such as when insurance agents handle auto insurance business for vehicles, they often need to take pictures of the vehicle VIN code or license plate for archiving, and entering these VIN codes or license plate information is often done manually, which is inefficient. Therefore, how to automatically identify these areas to further identify the VIN code or license plate, thereby improving image recognition efficiency, becomes the key. Summary of the invention
[0003] The embodiments of the present application provide an image processing method, device and computer-readable storage medium, which help to improve the recognition efficiency of VIN codes or license plates of vehicle images.
[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising:
[0005] Acquire a vehicle image, wherein the vehicle image is a VIN code image or a license plate image, wherein the VIN code image is an image including a VIN code area of the vehicle, and the license plate image is an image including a license plate area of the vehicle;
[0006] Identify the vehicle image according to a preset identification rule to identify the type of the vehicle image, where the type of the vehicle image is a VIN code image type or a license plate image type;
[0007] If it is determined that the type of the vehicle image is a VIN code image type, the VIN code image is processed according to a preset VIN code image processing rule to obtain the VIN code of the vehicle;
[0008] If it is determined that the type of the vehicle image is a license plate image type, the license plate image is processed according to a preset license plate image processing rule to obtain the license plate of the vehicle.
[0009] Optionally, the identifying the vehicle image according to a preset identification rule to identify the type of the vehicle image includes:
[0010] Inputting the vehicle image into a preset image detection model to obtain an output result of the image detection model, wherein the output result is used to indicate the type of the vehicle image;
[0011] In which, the image detection model is trained based on multiple pre-selected image samples and the type of each image sample, the multiple image samples include a first number of VIN code image samples and a second number of license plate image samples, and the absolute value of the difference between the first number and the second number does not exceed a preset number threshold.
[0012] Optionally, the processing the VIN code image according to a preset VIN code image processing rule to obtain the VIN code of the vehicle includes:
[0013] Performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation;
[0014] Performing edge detection on the VIN code image after the top-hat transformation to obtain a first edge image after edge detection;
[0015] Performing a morphological closing operation on the first edge image to obtain a first closed operation image;
[0016] Performing a morphological opening operation on the first closed operation image to obtain a first open operation image;
[0017] Determine a minimum bounding rectangle of a VIN code area in the first open operation image, and determine the area determined by the minimum bounding rectangle as the VIN code area included in the VIN code image;
[0018] The VIN code area is identified to identify the VIN code of the vehicle.
[0019] Optionally, before performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation, the method further includes:
[0020] Detecting whether the brightness of the VIN code image is within a preset brightness range;
[0021] If the brightness of the VIN code image is not within the brightness interval, the step of performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation is triggered.
[0022] Optionally, the structure element used in the morphological closing operation is a 1*N row vector structure element, where N is the number of columns and N is greater than 0;
[0023] Wherein, the N is obtained by setting the resolution of the VIN code image; and / or,
[0024] The N is obtained by setting the size of the VIN code area in the VIN code image, wherein the size includes length or width; and / or,
[0025] The N is set according to the ratio of the character spacing in the vin code area to the size of the vin code image; and / or,
[0026] The N is set according to the width of the vin code image.
[0027] Optionally, the processing the license plate image according to a preset license plate image processing rule to obtain the license plate of the vehicle includes:
[0028] Performing edge detection on the license plate image to obtain a second edge image after edge detection;
[0029] performing a morphological closing operation on the second edge image to obtain a second closed operation image;
[0030] performing a morphological opening operation on the second closed operation image to obtain a second open operation image;
[0031] Determine the minimum circumscribed rectangle of the license plate area in the second open operation image, and determine the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image;
[0032] The license plate area is identified to identify the license plate of the vehicle.
[0033] Optionally, the structure element used in the morphological closing operation is an M*N vector structure element, where M is the number of rows, N is the number of columns, and M and N are greater than 0;
[0034] Wherein, the M and N are obtained by setting the resolution of the license plate image; or,
[0035] The M and N are obtained by setting the size of the license plate area in the license plate image, wherein the size includes length and / or width; or,
[0036] The M and N are set according to the ratio of the character spacing in the license plate area to the size of the license plate image; or,
[0037] The M is obtained by setting according to the length of the license plate image, and N is obtained by setting according to the width of the license plate image.
[0038] In a second aspect, an embodiment of the present application provides an image processing device, which includes a unit for executing the method of the first aspect.
[0039] In a third aspect, an embodiment of the present application provides another image processing device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program that supports the image processing device to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect. Optionally, the image processing device may also include a user interface and / or a communication interface.
[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect above.
[0041] The embodiment of the present application can acquire a vehicle image and identify the type of the vehicle image according to a preset recognition rule, that is, determine whether the vehicle image is a VIN code image or a license plate image, and then can select the corresponding image processing rule according to the recognized type to process the vehicle image to obtain the VIN code or license plate included in the vehicle image. This improves the flexibility of image processing, helps to improve the recognition efficiency of the VIN code or license plate of the vehicle image, and improves the image processing effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0043] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;
[0044] Figure 2 is a flowchart of another image processing method provided in an embodiment of the present application;
[0045] Figure 3 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0046] Figure 4 It is a structural schematic diagram of another image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0047] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0048] The technical solution of the present application can be applied to an image processing device, which may include various terminals, servers or other devices for processing images, such as extracting the VIN code area in an image, identifying the VIN code in an image, etc. The terminal involved in the present application may be a mobile phone, a computer, a tablet, a personal computer, a smart watch, etc., and the present application does not limit it.
[0049] The present application can identify whether the acquired vehicle image is a VIN code image or a license plate image, and then can select the corresponding image processing rules according to the identification result to perform image processing on the vehicle image to obtain the VIN code or license plate included in the vehicle image, which improves the flexibility of image processing and helps to improve the recognition efficiency of the VIN code or license plate of the vehicle image and the image processing effect. The following are detailed descriptions.
[0050] See also Figure 1 , Figure 1 is a flow chart of an image processing method provided by an embodiment of the present application. Specifically, the method of this embodiment can be specifically applied to the above-mentioned image processing device. Figure 1 As shown, the image processing method may include the following steps:
[0051] 101. Obtain a vehicle image.
[0052] The vehicle image may refer to an image including a vehicle, and the vehicle image may specifically be a VIN code image or a license plate image. The VIN code image may be an image including the VIN code area of the vehicle, and the license plate image may be an image including the license plate area of the vehicle. VIN (Vehicle Identification Number) can be called a vehicle identification number, or a frame number, etc. The VIN code is a set of seventeen numbers that are used to identify a unique number of a vehicle, that is, each vehicle has a unique VIN code. The VIN code can be used to identify the manufacturer, engine, chassis serial number, and other performance information of the vehicle.
[0053] Optionally, the vehicle image may be acquired in real time by a preset shooting device, or may be acquired by the shooting device and stored in a preset queue or database to be processed, and then acquired from the queue or database, or may be a vehicle image received from other devices, etc. This application does not limit the method for acquiring the vehicle image. The shooting device may be a camera, a video camera, a camera, or other device or equipment that can be used for shooting, and the shooting device may be set in the image processing device, or may be connected to the image processing device.
[0054] 102. Identify the vehicle image according to a preset identification rule to identify the type of the vehicle image, where the type of the vehicle image is a VIN code image type or a license plate image type.
[0055] Among them, the type of the vehicle image is the vin code image type, which can be used to indicate that the vehicle image is a vin code image, and the type of the vehicle image is the license plate image type, which can be used to indicate that the vehicle image is a license plate image.
[0056] Specifically, one or more recognition rules may be preset. For example, the recognition rule may refer to a rule for inputting the vehicle image into a preset image detection model to identify the type of the vehicle image, that is, a recognition rule based on an image detection model. For another example, the recognition rule may also refer to a rule for taking the type corresponding to the vehicle image with the highest similarity to the vehicle image among the vehicle image samples stored in the database as the type of the vehicle image, that is, a recognition rule based on similarity comparison; that is, after acquiring the vehicle image, the image processing device may determine the vehicle image with the highest similarity to the vehicle image among the vehicle image samples stored in the database, and determine the type corresponding to the vehicle image with the highest similarity as the type of the acquired vehicle image, wherein the database may store each vehicle image sample and the type (label) corresponding to each vehicle image sample. For another example, the recognition rule may also refer to a rule for determining the type of the vehicle image according to the source label of the vehicle image, that is, an recognition rule based on the source label; that is, the image processing device may determine the rule of the vehicle image according to the source of the acquired vehicle image. For example, if the vehicle image carries a source label, assuming that the source label indicates that the vehicle image comes from a vin code image database, the type of the vehicle image may be determined to be a vin code image type; assuming that the source label indicates that the vehicle image comes from a license plate image database, the type of the vehicle image may be determined to be a license plate image type; for another example, different shooting devices are used to capture vin code images and license plate images respectively. If the source label indicates that the vehicle image comes from a shooting device that captures vin code images (such as the source label is the identification of the shooting device of the vin code image), the type of the vehicle image may be determined to be a vin code image type; if the source label indicates that the vehicle image comes from a shooting device that captures license plate images (such as the source label is the identification of the shooting device of the license plate image), the type of the vehicle image may be determined to be a license plate image type, and so on. They are not listed one by one here.
[0057] Optionally, the recognition rules used by different image processing devices may be different, and / or the same image processing device may also switch the recognition rules according to a preset switching rule to identify the image type, so as to improve the flexibility and reliability of type determination. The switching rule may include a switching rule for switching the recognition rule according to a preset time interval; or may include a switching rule for switching the recognition rule according to the label of the vehicle image (such as an importance level label, a source label, etc.); or may include a switching rule for switching the recognition rule according to a switching instruction input by the user (switching in sequence, or the switching instruction may carry information about the recognition rule to be switched to); or may include a switching rule for switching the recognition rule when the number of errors in the identified VIN code or license plate reaches a preset number (or the recognition error frequency in a preset time period is greater than a preset frequency threshold, etc.), etc. Specifically, the corresponding relationship between the switching rule and the recognition rule can be set first. This helps to improve the flexibility and reliability of type recognition. For example, the image processing device may switch the recognition rules at preset time intervals, such as every 24 hours; for another example, the image processing device may switch the recognition rules according to the source label of the vehicle. For example, when the importance level label indicates that the level of the vehicle image is important, the recognition rules based on the image detection model may be adopted; and when the importance level label indicates that the level of the vehicle image is general, the recognition rules based on the source label may be adopted, and so on. They are not listed here one by one.
[0058] 103. If it is determined that the type of the vehicle image is a VIN code image type, the VIN code image is processed according to a preset VIN code image processing rule to obtain the VIN code of the vehicle.
[0059] That is to say, when the type of the vehicle image is a vin code image type, it can be determined that the vehicle image is a vin code image, and then the vin code image can be processed according to a preset vin code image processing rule to obtain the vin code included in the vin code image, that is, the vin code of the vehicle.
[0060] Optionally, the image processing device processes the vin code image according to a preset vin code image processing rule to obtain the vin code of the vehicle, which may refer to: performing edge detection on the vin code image to obtain a first edge image after edge detection; performing a morphological closing operation on the first edge image to obtain a first closed operation image; performing a morphological opening operation on the first closed operation image to obtain a first open operation image; determining the minimum circumscribed rectangle of the vin code area in the first open operation image, and determining the area determined by the minimum circumscribed rectangle as the vin code area included in the vin code image; identifying the vin code area to identify the vin code included in the vin code image, that is, identifying the vin code of the vehicle. Further optionally, before edge detection, the image processing device may also perform a tophat transformation on the vin code image to obtain a vin code image after tophat transformation, so that when edge detection is performed, edge detection may be performed on the vin code image after tophat transformation to obtain the first edge image after edge detection, and subsequent processes may be executed. Among them, the tophat transformation can be used to detect bright details in dark places and extract areas with higher grayscale in images.
[0061] 104. If it is determined that the type of the vehicle image is a license plate image type, the license plate image is processed according to a preset license plate image processing rule to obtain the license plate of the vehicle.
[0062] That is to say, when the type of the vehicle image is a license plate image type, it can be determined that the vehicle image is a license plate image, and then the license plate image can be processed according to a preset license plate image processing rule to obtain the license plate included in the license plate image, that is, the vehicle license plate.
[0063] Optionally, the image processing device processes the license plate image according to a preset license plate image processing rule to obtain the license plate included in the license plate image, which may refer to: performing edge detection on the license plate image to obtain a second edge image after edge detection; performing a morphological closing operation on the second edge image to obtain a second closed operation image; performing a morphological opening operation on the second closed operation image to obtain a second open operation image; determining the minimum circumscribed rectangle of the license plate area in the second open operation image, and determining the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image; and identifying the license plate area to identify the license plate included in the license plate image, that is, the license plate of the vehicle.
[0064] In this embodiment, the image processing device can acquire a vehicle image and identify the type of the vehicle image according to a preset recognition rule, that is, determine whether the vehicle image is a VIN code image or a license plate image, and then can select the corresponding image processing rule according to the recognized type to process the vehicle image to obtain the VIN code or license plate included in the vehicle image. This improves the flexibility of image processing, helps to improve the recognition efficiency of the VIN code or license plate of the vehicle image, and improves the image processing effect.
[0065] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another image processing method provided in an embodiment of the present application. Specifically, Figure 2 As shown, the image processing method may include the following steps:
[0066] 201. Obtain a vehicle image.
[0067] The vehicle image may be a VIN code image or a license plate image, the VIN code image may be an image including the VIN code area of the vehicle, and the license plate image may be an image including the license plate area of the vehicle. There may be multiple ways to obtain the vehicle image, which will not be described in detail here.
[0068] 202. Input the vehicle image into a preset image detection model to obtain an output result of the image detection model, where the output result is used to indicate the type of the vehicle image.
[0069] Among them, the type of the vehicle image is a vin code image type or a license plate image type. The vin code image type can be used to indicate that the vehicle image is a vin code image, and the license plate image type can be used to indicate that the vehicle image is a license plate image.
[0070] The image detection model can be obtained by training based on a plurality of pre-selected image samples and the type of each image sample. Optionally, the plurality of image samples may include a first number of vin code image samples and a second number of license plate image samples, and the absolute value of the difference between the first number and the second number does not exceed a preset number threshold. That is, the image processing device can select a first number of vin code image samples and a second number of license plate image samples, that is, select image samples of different types with a balanced number to perform model training, thereby improving the model training effect. In some embodiments, it may be encountered that the vehicle license plate or vin code is collected separately. The image processing device can determine different image processing rules according to different regional images by detecting whether the acquired vehicle image is a license plate image or a vin code image, and identify according to the corresponding image processing rules, including license plate number recognition of the license plate area (license plate image processing rules) or vin code recognition of the vin code area (vin code image processing rules), etc. In a possible implementation, as described above, an image detection model can be established based on each license plate image in the historical data and each vin code image in the historical data, and the vehicle image acquired subsequently can be identified as a license plate area or a vin code area by inputting the image detection model.
[0071] Optionally, the output result may include the type of the vehicle image, or may include an identifier for indicating the type of the vehicle image, such as "0" representing a VIN code image type, "1" representing a license plate image type, and so on, which is not limited in this application.
[0072] Further optionally, if the vehicle image is captured by a photographing device, the vehicle environment information of the vehicle can also be obtained when the vehicle image is captured, and then the target shooting parameters for photographing the vehicle are determined based on the vehicle environment information, and the vehicle image is captured using the photographing device and according to the target shooting parameters to obtain the vehicle image. Optionally, the vehicle can refer to any vehicle within the shooting range of the photographing device, or can be a vehicle of a specific model within the shooting range, or can be a vehicle in a specific location area within the shooting range, or can be a vehicle that sends an indication signal (such as turning on the double flash or turning on the wipers, etc.) within the shooting range, etc., and the selection rules for the vehicle can be pre-set, thereby improving the flexibility of vehicle selection. Further optionally, the vehicle environment information may include any one or more of the vehicle model, the ambient light intensity of the vehicle, the distance between the vehicle and the shooting device (or the distance between the vehicle and the image processing device), the type of the vehicle image (for example, the type identified by the previous image detection model is used as the type of the vehicle image collected subsequently, or the type preset by the user is used as the type of the vehicle image collected, etc., and the vehicle image collected subsequently may directly trigger the corresponding image processing process without the need for type identification), and the system time. Further optionally, the vehicle environment information may be input by the user; or it may be automatically identified by the image processing device, such as determining the vehicle model by previewing the collected image of the vehicle or by identifying the vehicle logo, determining the distance by a preset distance sensor, determining the ambient light intensity by a preset light sensor, determining the system time by a preset time module, etc.; or it may be sent to the image processing device after other devices (such as the vehicle) identify the vehicle environment information, and the image processing device may receive the vehicle environment information of the vehicle sent from other devices, etc., which are not listed here one by one.
[0073] Optionally, when determining the target shooting parameters for photographing the vehicle, the image processing device may be based on the preset correspondence between each vehicle environment information and the shooting parameters, and based on the correspondence, quickly determine the shooting parameters corresponding to the vehicle environment information as the target shooting parameters to improve the efficiency of determining the shooting parameters; or, the image processing device may also be based on the vehicle environment information such as the vehicle model, based on the similarity between the captured VIN code image of the vehicle and each VIN code image sample under the vehicle model stored in the database, and use the shooting parameters corresponding to the VIN code image sample under the vehicle model in the database with the highest similarity to the VIN code image of the vehicle as the target shooting parameters of the vehicle, and the database may store multiple VIN code image samples, multiple vehicle models, and (preferable) shooting parameters corresponding to each VIN code image sample and each vehicle model; or, the image processing device may also be based on the vehicle environment information and use the shooting parameters of the vehicle that is the same as the vehicle environment information that was manually adjusted last time as the target shooting parameters, and so on, which are not listed here one by one. Further optionally, the shooting parameters involved in the present application, such as the target shooting parameters, may include shooting angle (controlling the shooting device to shoot at the shooting angle), focal length, aperture, ISO and / or EV value, etc., which are not limited in the present application.
[0074] Optionally, in a possible implementation, the vehicle environment information of the vehicle may include the model of the vehicle. When determining the target shooting parameters, the image processing device may search for shooting parameters corresponding to a vehicle model that is the same as the vehicle model from a database based on a preset correspondence between the vehicle model and the shooting parameters, and determine the searched shooting parameters as the target shooting parameters for shooting the vehicle. Among them, the database may pre-store a variety of vehicle models and shooting parameters corresponding to each vehicle model. The shooting parameters corresponding to each vehicle model may be shooting parameters of vehicle image samples selected for the model with better shooting effects, such as shooting quality greater than a preset quality threshold and / or the proportion of the vin code area (or license plate area) in the image greater than a preset proportion threshold, etc.
[0075] Optionally, in a possible implementation, the vehicle environment information of the vehicle includes the vehicle model and the ambient light intensity of the vehicle. When determining the target shooting parameters, the image processing device may determine the target ambient light intensity interval in which the ambient light intensity of the vehicle is located among the multiple ambient light intensity intervals based on the preset multiple ambient light intensity intervals; according to the correspondence between the preset vehicle model, ambient light intensity interval and shooting parameters, the shooting parameters corresponding to the vehicle model and the target ambient light intensity interval are searched from the database, and the searched shooting parameters are determined as the target shooting parameters for shooting the vehicle. Among them, the database may pre-store multiple vehicle models, the multiple ambient light intensity intervals and the shooting parameters corresponding to each vehicle model and each ambient light intensity interval.
[0076] Optionally, in a possible implementation, the vehicle environment information of the vehicle includes the vehicle model and the distance between the vehicle and the shooting device. When determining the target shooting parameter, the image processing device may determine the target distance interval in which the distance between the vehicle and the shooting device is located in the multiple distance intervals according to the preset multiple distance intervals; according to the correspondence between the preset vehicle model, distance interval and shooting parameters, the shooting parameters corresponding to the vehicle model and the target distance interval are searched from the database, and the searched shooting parameters are determined as the target shooting parameters for shooting the vehicle. Among them, the database may pre-store multiple vehicle models, the multiple distance intervals and the shooting parameters corresponding to each vehicle model and each distance interval.
[0077] Optionally, in a possible implementation, the vehicle environment information of the vehicle includes the vehicle model and system time of the vehicle. When determining the target shooting parameter, the image processing device may determine the target time period to which the system time belongs in the multiple time periods according to the preset multiple time periods; according to the correspondence between the preset vehicle model, time period and shooting parameter, the shooting parameter corresponding to the vehicle model and the target time period is searched from the database, and the searched shooting parameter is determined as the target shooting parameter for shooting the vehicle. The database may pre-store multiple vehicle models, the multiple time periods and the shooting parameters corresponding to each vehicle model and each time period.
[0078] Optionally, in a possible implementation, the vehicle environment information of the vehicle includes the system time, and the image processing device may determine the shooting parameters corresponding to the system time as the target shooting parameters according to the correspondence between the preset time period and the shooting parameters when determining the target shooting parameters; or, the vehicle environment information of the vehicle includes the ambient light intensity of the vehicle, and the image processing device may determine the shooting parameters corresponding to the ambient light intensity as the target shooting parameters according to the correspondence between the preset ambient light intensity interval and the shooting parameters when determining the target shooting parameters; the vehicle environment information of the vehicle includes the distance between the vehicle and the shooting device, and the image processing device may determine the shooting parameters corresponding to the distance as the target shooting parameters according to the correspondence between the preset distance interval and the shooting parameters when determining the target shooting parameters; or, the vehicle environment information may also include the vehicle model, the ambient light intensity of the vehicle, the distance between the vehicle and the shooting device, and the system time, or include the ambient light intensity of the vehicle and the distance between the vehicle and the shooting device, or include the ambient light intensity of the vehicle and the system time, and so on, which are not listed here one by one.
[0079] For example, the camera device may be mounted at a fixed position, and the camera device may rotate, that is, the camera angle of the camera device may be adjusted. Thus, the vin code image data of different vehicle models may be obtained by analyzing the vin code images in the historical data collected by the camera device, and a vin code image with better shooting effect (such as covering the entire vin code area and with less interference) may be selected therefrom, and the shooting parameters of the vin code image may be obtained, so that after the camera device identifies the vehicle model, it can quickly call the shooting parameters according to the vehicle model, including the shooting angle of the camera device (control the camera device to shoot at the shooting angle), focal length, aperture, ISO, EV value, etc. Alternatively, the license plate image in the collected historical data may be analyzed to obtain the license plate image data of different vehicle models, and a license plate image with better shooting effect may be selected therefrom, and the shooting parameters of the license plate image may be obtained, so that after the camera device identifies the vehicle model, it can quickly call the shooting parameters according to the vehicle model to shoot the license plate image. Specifically, when a shooting instruction for a regional image is received, it can be determined that the shooting instruction corresponds to a license plate image shooting instruction or a VIN code image shooting instruction to trigger the call of corresponding shooting parameters for shooting, and it can be determined based on the shooting instruction whether the acquired captured image is a license plate image or a VIN code image, and then the corresponding image processing rules can be determined.
[0080] For another example, the shooting time and shooting parameters of the VIN code image (or license plate image) with better shooting effect can be obtained by combining the time and / or the ambient light intensity, and then the more matching shooting parameters can be quickly called by obtaining the current vehicle model and the current time (and / or the ambient light intensity) to improve the shooting reliability. Different times or different ambient light intensities will affect the shooting effect. For example, the image effects obtained by shooting with the same shooting parameters during the day and at night are completely different. Therefore, the corresponding relationship between different time periods and vehicle models (can also be combined with the identified vehicle image type, that is, the image area type) and the shooting parameters of the shooting device can be preset; or the corresponding relationship between the ambient light intensity and the vehicle model (can also be combined with the identified vehicle image type) and the shooting parameters of the shooting device can be preset, or the corresponding relationship between the ambient light intensity and the vehicle model (can also be combined with the identified vehicle image type) and the shooting parameters of the shooting device can be preset, or the corresponding relationship between different time periods, ambient light intensities and vehicle models (can also be combined with the identified vehicle image type) and the shooting parameters of the shooting device can be preset, and so on. For example, the VIN code images in the historical data collected by the shooting device can be analyzed (each VIN code image is associated with the shooting time when the image was shot) to obtain VIN code image data of different time periods, and the VIN code image with better shooting effect can be selected according to the VIN code image data of the same vehicle model in each time period, and the shooting parameters of the VIN code image can be obtained, so as to determine the correspondence between different time periods and vehicle models and the shooting parameters of the shooting device; so that after the shooting device recognizes the time period and vehicle model of the current time, it can quickly call the corresponding shooting parameters for shooting according to the recognized time period and vehicle model. For another example, the vin code images in the historical data collected by the shooting device can be analyzed (each vin code image is associated with the ambient light intensity when the image was shot), and the vin code image data under different ambient light intensities can be obtained. According to the vin code image data of the same vehicle model under each ambient light intensity interval, the vin code image with better shooting effect can be selected, and the shooting parameters of the vin code image can be obtained, so as to determine the correspondence between different ambient light intensity intervals and vehicle models, and the shooting parameters of the shooting device; so that after the shooting device recognizes the ambient light intensity interval in which the current ambient light intensity is located and the vehicle model, it can quickly call the corresponding shooting parameters for shooting according to the recognized ambient light intensity interval and vehicle model. Combined with information such as vehicle model time, the shooting parameters can be quickly called, which improves the shooting quality, and then improves the processing effect of the subsequent vin code area or license plate area.
[0081] 203. If the type of the vehicle image is a VIN code image type, it is determined that the vehicle image is a VIN code image, and steps 205-207 may be executed.
[0082] After determining that the acquired vehicle image is a VIN code image, the VIN code image can be processed to extract the VIN code area. When processing the VIN code image, a set of simple and fast algorithms such as tophat transformation, edge detection, morphological closing operation, morphological opening operation, and calculation of the minimum enclosing rectangle can be used to determine the VIN code area and then identify the VIN code.
[0083] 204. If the type of the vehicle image is a license plate image type, it is determined that the vehicle image is a license plate image, and steps 208-210 may be executed.
[0084] After determining that the acquired vehicle image is a license plate image, the license plate image can be processed to extract the license plate area. When processing the license plate image, a set of simple and fast algorithms such as edge detection, morphological closing operation, morphological opening operation, and calculation of the minimum circumscribed rectangle can be used to determine the license plate area and then identify the license plate.
[0085] 205. Perform a tophat transformation on the VIN code image to obtain a VIN code image after the tophat transformation, and perform edge detection on the VIN code image after the tophat transformation to obtain a first edge image after edge detection.
[0086] By performing a tophat transformation on the captured VIN code image, the interference to the VIN code image area can be reduced. Since tophat is effective in detecting bright details in dark areas in image segmentation, especially targets with uniform width or size, the VIN code area can be further distinguished by performing a tophat transformation on the captured VIN code image. Optionally, there are some vehicle body reflection interferences in the VIN code image. In order to reduce these interferences, the original VIN code image, that is, the original image, can be converted into a grayscale image. After obtaining the converted grayscale image, the grayscale image is tophat transformed to obtain the tophat transformation result. After the original VIN code image is filtered by tophat, it has a simple and uniform background and foreground, so as to better distinguish the VIN code area.
[0087] For example, the tophat transform of a grayscale image f is defined as f minus the result of its opening operation, that is, That(f)=f-(f°b), where b is the mean filter template and ° is the opening operation.
[0088] Optionally, before performing a tophat transformation on the vin code image to obtain a vin code image after the tophat transformation, the image processing device may also detect whether the brightness of the vin code image is within a preset brightness range; if the brightness of the vin code image is not within the brightness range, the step of performing a tophat transformation on the vin code image to obtain a vin code image after the tophat transformation may be triggered. Further, if the brightness of the vin code image is within the brightness range, that is, when the shooting conditions are good, the tophat transformation may not be performed, and the vin code image may be directly subjected to edge detection, thereby helping to reduce equipment overhead.
[0089] Furthermore, when performing edge detection, a preset edge detection algorithm such as the Canny operator can be used for edge detection. The characteristic of the Canny algorithm is that it attempts to assemble candidate pixels of independent edges into contours. The specific algorithm steps for finding edge points using the Canny operator are as follows:
[0090] 1) Use a Gaussian filter to convolve with the image (such as the VIN code image) to smooth the image and filter out noise;
[0091] 2) Use the first-order partial derivative finite difference to calculate the gradient amplitude (gradient strength) and direction of each pixel in the image; the edges in the image can point to various directions. The Canny algorithm can use operators (such as Roberts, Prewitt, Sobel, etc.) to detect horizontal, vertical and diagonal edges in the image, obtain the first-order derivative values in the horizontal and vertical directions, and thus determine the gradient amplitude and direction of the pixel point. The calculation of the gradient direction and the selection of the gradient operator are consistent;
[0092] 3) Non-maximum suppression is performed on the gradient amplitude to eliminate the stray response caused by edge detection; specifically, when non-maximum suppression is performed on each pixel in the gradient image, the gradient strength of the current pixel can be compared with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is retained as an edge point, otherwise the pixel point will be suppressed, thereby suppressing all gradient values outside the local maximum value to 0, and realizing a "thin" edge;
[0093] 4) Use the double-threshold algorithm to detect and connect edges; specifically, after 3), there are still some edge pixels caused by noise and color changes. Therefore, by setting high and low thresholds, if the gradient value of the edge pixel is higher than the high threshold, it can be marked as a strong edge pixel; if the gradient value of the edge pixel is less than the high threshold and greater than the low threshold, it can be marked as a weak edge pixel; if the gradient value of the edge pixel is less than the low threshold, it is suppressed. Thus, edge detection can be used to extract useful structural information from different visual objects and reduce the amount of data to be processed.
[0094] 206. Perform a morphological closing operation on the first edge image to obtain a first closed operation image, and perform a morphological opening operation on the first closed operation image to obtain a first open operation image; determine the minimum bounding rectangle of the VIN code area in the first open operation image, and determine the area determined by the minimum bounding rectangle as the VIN code area included in the VIN code image.
[0095] After edge detection is performed on the VIN code image to obtain the edge detection result image, i.e., the edge image, a morphological closing operation can be performed on the edge graph, first dilation and then erosion. Thus, a morphological closing operation result is obtained to exclude small black holes (black areas). When performing a morphological closing operation, the structure element used can be a row vector structure element with a size of 1*N to connect discontinuous edges in the horizontal direction.
[0096] Among them, the structural element used in the morphological closing operation can be a 1*N row vector structural element, N can be the number of columns, which can be used to indicate the width of the structural element, and N is greater than 0. Optionally, the N can be set according to the resolution of the vin code image; and / or, the N can be set according to the size (size) of the vin code area in the vin code image, and the size includes length or width; and / or, the N can be set according to the ratio of the character spacing in the vin code area to the size of the vin code image; and / or, the N can be set according to the width of the vin code image. For example, the N can be set according to the resolution of the image and / or the size of the vin code area in the image. The larger the resolution and / or the larger the vin code area, the larger the value of N. Specifically, the corresponding relationship between N and the resolution of the image (and / or the size of the vin code area in the image) can be pre-set. For another example, N can be obtained based on the statistical relationship between the character spacing size of the vin code area in the captured image and the vin code image size. For example, the larger the ratio of the character spacing size to the vin code image size, the larger the value of N. Specifically, the corresponding relationship between N and the ratio of the character spacing size to the image size can be pre-set. For another example, N can be obtained based on the image width setting, for example, the value of N can be 5% of the image width. Thus, the morphological closing operation result is obtained to exclude small black holes (black areas). In other words, the structural element N for the closing operation can be obtained based on the resolution of the image and / or the size of the vin code area in the image, the ratio of the character spacing size of the vin code area in the image to the image size, the image width and other parameters, which is highly flexible and improves the image processing effect.
[0097] Furthermore, after the morphological closing operation, the morphological opening operation is performed, first erosion and then expansion, to remove small-size noise in the image. The structural element used in the morphological opening operation can be consistent with the structural element used in the above-mentioned closing operation. Thus, the morphological opening operation result is obtained to achieve the effects of eliminating small objects, separating objects at thin points, and smoothing the boundaries of larger objects without significantly changing their areas.
[0098] Furthermore, after the opening operation, in the result image, i.e. the opening operation image, the white part displayed is the foreground, and the black part is the background. Therefore, by scanning the result image, i.e. the opening operation image, the minimum enclosing rectangle of the foreground part can be calculated. The area determined (enclosed) by the minimum enclosing rectangle is the area where the VIN code sign is located, i.e. the VIN code area.
[0099] 207. Identify the VIN code area to identify the VIN code included in the VIN code image.
[0100] After the VIN code area is extracted, the image of the VIN code area can be stored, and / or the characters (signs) or barcodes included in the VIN code area can be further identified. Optionally, the VIN code area can be identified by OCR character recognition or other recognition methods, which are not limited in this solution.
[0101] Optionally, in order to ensure the reliability of the identified VIN code, after the VIN code is identified, the VIN code may be verified to determine whether it is correct / legal, for example, the 9th bit of the identified VIN code may be verified. Because the 9th bit of the VIN code is always the check bit, the correct check bit can be obtained by performing a series of calculations on the other bits in the VIN code (where the 9th bit of the VIN code is any number from 1 to 9 or "X". The calculation method can be to multiply the corresponding value of each remaining letter or number in the VIN by the weighting coefficient of the bit, and then divide it by 11. The remainder is the check bit. If the remainder is 10, the check bit is "X". The weighting coefficient can be found in the "World Vehicle Identification Code (VIN) Data Manual" P21~23. For example, first convert the letters in the other 16 bits into numbers according to the following relationship: A=1B=2C=3D=4E=5F=6G=7H=8J=1K=2L=3M=4N=5P=7R=9S=2T=3U=4V=5W=6X=7Y=8Z=9, and each position and its corresponding weighting coefficient are: Position: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17, weighting coefficient: 8 7 6 5 4 32 10*9 8 7 6 5 4 3 2), and then the calculated check bit can be compared with the 9th bit to determine whether the two are the same, and whether the recognized vin code has an error, that is, when the two are different, it indicates that the vin code has an error. Alternatively, it can be detected whether there are illegal characters in the recognized vin code, and specifically, a matching comparison can be performed by presetting a legal character whitelist. If there are illegal characters, such as characters outside the whitelist, such as I, O, Q, etc., it can be determined that the recognized vin code has an error. Alternatively, the number of digits of the recognized vin code can be detected, because the normal vin code is fixed to 17 digits, if it is longer or shorter than 17 digits, it can be determined that the recognized vin code has an error. If it is determined that the recognized vin code has an error, re-recognition can also be triggered (if the number of errors reaches a preset number of thresholds, such as 3 times, the vin code area can be re-extracted and re-recognized), or an alarm message can be issued to prompt manual recognition or detection, etc.
[0102] After obtaining the correct VIN code, you can store the VIN code, or handle vehicle business such as car insurance based on the VIN code, thereby improving the intelligence and efficiency of business handling.
[0103] 208. Perform edge detection on the license plate image to obtain a second edge image after edge detection.
[0104] 209. Perform a morphological closing operation on the second edge image to obtain a second closed operation image, perform a morphological opening operation on the second closed operation image to obtain a second open operation image, determine the minimum circumscribed rectangle of the license plate area in the second open operation image, and determine the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image.
[0105] The structural element used in the morphological closing operation may be an M*N vector structural element, where M may be the number of rows, which may be used to indicate the length (height) of the structural element; N may be the number of columns, which may be used to indicate the width of the structural element, and both M and N are greater than 0. Optionally, the M and N may be set according to the resolution of the license plate image; and / or, the M and N may be set according to the size of the license plate area in the license plate image, the size including the length (height) and / or the width; and / or, the M and N may be set according to the ratio of the character spacing in the license plate area to the size of the license plate image (such as the height or the width); and / or, the M may be set according to the height of the license plate image, and the N may be set according to the width of the license plate image, such as the M being 5% of the height and the N being 5% of the width.
[0106] 210. Identify the license plate area to identify the license plate included in the license plate image.
[0107] After the license plate area is extracted, the image of the license plate area can be stored, and / or the characters (license plate) included in the license plate area can be further identified. Optionally, the VIN code area can be identified by OCR character recognition or other recognition methods, which are not limited in this solution.
[0108] Optionally, the description of steps 208-210 may refer to the relevant description of steps 205-207, which will not be repeated here.
[0109] The VIN code area or license plate area extraction method of the present application improves the detection efficiency when extracting the VIN code area or license plate in the image, can well overcome the interference of the reflected scenery of the vehicle body, has high accuracy, and can be applied in embedded systems with limited performance, with a wide range of applications.
[0110] In this embodiment, the image processing device can obtain a vehicle image and input the vehicle image into a preset image detection model to identify the type of the vehicle image, that is, determine whether the vehicle image is a VIN code image or a license plate image, and then can select the corresponding image processing rules to process the vehicle image according to the identified type, including selecting tophat transformation, edge detection, morphological closing operation, morphological opening operation, calculation of the minimum circumscribed rectangle, etc., to obtain the VIN code or license plate included in the vehicle image, which improves the flexibility of image processing, helps to improve the recognition efficiency of the VIN code or license plate of the vehicle image, and improves the image processing effect.
[0111] The above method embodiments are all examples of the image processing method of the present application. The description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0112] See also Figure 3 , Figure 3 300 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device in the embodiment of the present application includes a unit for executing the above-mentioned image processing method. Specifically, the image processing device 300 in this embodiment may include: an acquisition unit 301 and a processing unit 302. Among them,
[0113] The acquisition unit 301 is used to acquire a vehicle image, wherein the vehicle image is a VIN code image or a license plate image, wherein the VIN code image is an image including a VIN code area of the vehicle, and the license plate image is an image including a license plate area of the vehicle;
[0114] The processing unit 302 is used to identify the vehicle image according to a preset recognition rule to identify the type of the vehicle image, where the type of the vehicle image is a VIN code image type or a license plate image type;
[0115] The processing unit 302 is further configured to process the VIN code image according to a preset VIN code image processing rule to obtain the VIN code of the vehicle when the type of the vehicle image is a VIN code image type;
[0116] The processing unit 302 is further configured to process the license plate image according to a preset license plate image processing rule to obtain the license plate of the vehicle when the type of the vehicle image is a license plate image type.
[0117] Optionally, the processing unit 302 may be specifically configured to input the vehicle image into a preset image detection model to obtain an output result of the image detection model, wherein the output result is used to indicate the type of the vehicle image;
[0118] In which, the image detection model is trained based on multiple pre-selected image samples and the type of each image sample, the multiple image samples include a first number of VIN code image samples and a second number of license plate image samples, and the absolute value of the difference between the first number and the second number does not exceed a preset number threshold.
[0119] Optionally, the processing unit 302 can be specifically used to perform a tophat transform on the VIN code image to obtain a VIN code image after the tophat transform; perform edge detection on the VIN code image after the tophat transform to obtain a first edge image after edge detection; perform a morphological closing operation on the first edge image to obtain a first closed operation image; perform a morphological opening operation on the first closed operation image to obtain a first open operation image; determine the minimum enclosing rectangle of the VIN code area in the first open operation image, and determine the area determined by the minimum enclosing rectangle as the VIN code area included in the VIN code image; identify the VIN code area to identify the VIN code of the vehicle.
[0120] Optionally, the structure element used in the morphological closing operation performed on the first edge image is a 1*N row vector structure element, where N is the number of columns and N is greater than 0;
[0121] Wherein, the N is obtained by setting the resolution of the VIN code image; and / or,
[0122] The N is obtained by setting the size of the VIN code area in the VIN code image, wherein the size includes length or width; and / or,
[0123] The N is set according to the ratio of the character spacing in the vin code area to the size of the vin code image; and / or,
[0124] The N is set according to the width of the vin code image.
[0125] Optionally, the processing unit 302 can be specifically used to perform edge detection on the license plate image to obtain a second edge image after edge detection; perform a morphological closing operation on the second edge image to obtain a second closed operation image; perform a morphological opening operation on the second closed operation image to obtain a second open operation image; determine the minimum circumscribed rectangle of the license plate area in the second open operation image, and determine the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image; identify the license plate area to identify the license plate included in the license plate image.
[0126] Optionally, the structure element used in the morphological closing operation performed on the second edge image is an M*N vector structure element, where M is the number of rows, N is the number of columns, and M and N are greater than 0;
[0127] Wherein, the M and N are obtained by setting the resolution of the license plate image; and / or,
[0128] The M and N are obtained by setting the size of the license plate area in the license plate image, wherein the size includes length and / or width; and / or,
[0129] The M and N are set according to the ratio of the character spacing in the license plate area to the size of the license plate image; and / or,
[0130] The M is obtained by setting according to the length of the license plate image, and N is obtained by setting according to the width of the license plate image.
[0131] Optionally, the processing unit 302 can also be used to detect whether the brightness of the VIN code image is within a preset brightness range before performing a tophat transformation on the VIN code image to obtain a VIN code image after the tophat transformation; if the brightness of the VIN code image is not within the brightness range, triggering the tophat transformation of the VIN code image to obtain a VIN code image after the tophat transformation.
[0132] Specifically, the image processing device can realize the above-mentioned Figure 1 to Figure 2 Part or all of the steps in the image processing method in the illustrated embodiment. It should be understood that the embodiment of the present application is an apparatus embodiment corresponding to the method embodiment, and the description of the method embodiment is also applicable to the embodiment of the present application.
[0133] See also Figure 4 , Figure 4 is a schematic diagram of the structure of another image processing device provided in an embodiment of the present application. The image processing device is used to execute the above method. Figure 4 As shown, the image processing device 400 in this embodiment may include: one or more processors 401 and a memory 402. The image processing device may also include a shooting device or be connected to the shooting device. Optionally, the image processing device may also include one or more user interfaces 403, and / or one or more communication interfaces 404. The processor 401, user interface 403, communication interface 404 and memory 402 may be connected via a bus 405, or may be connected in other ways. Figure 4The bus mode is used as an example in the description. The memory 402 is used to store a computer program, the computer program includes program instructions, and the processor 401 is used to execute the program instructions stored in the memory 402.
[0134] Among them, the processor 401 can be used to call the program instructions to perform the following steps: obtain a vehicle image, the vehicle image is a VIN code image or a license plate image, the VIN code image is an image including the VIN code area of the vehicle, and the license plate image is an image including the license plate area of the vehicle; identify the vehicle image according to a preset recognition rule to identify the type of the vehicle image, the type of the vehicle image is a VIN code image type or a license plate image type; if the type of the vehicle image is a VIN code image type, process the VIN code image according to a preset VIN code image processing rule to obtain the VIN code of the vehicle; if the type of the vehicle image is a license plate image type, process the license plate image according to a preset license plate image processing rule to obtain the license plate of the vehicle.
[0135] Optionally, when performing the step of identifying the vehicle image according to a preset recognition rule to identify the type of the vehicle image, the process 401 may specifically perform the following steps: inputting the vehicle image into a preset image detection model to obtain an output result of the image detection model, wherein the output result is used to indicate the type of the vehicle image;
[0136] In which, the image detection model is trained based on multiple pre-selected image samples and the type of each image sample, the multiple image samples include a first number of VIN code image samples and a second number of license plate image samples, and the absolute value of the difference between the first number and the second number does not exceed a preset number threshold.
[0137] Optionally, when the processor 401 processes the VIN code image according to a preset VIN code image processing rule to obtain the VIN code included in the VIN code image, the processor 401 may specifically perform the following steps: performing a tophat transformation on the VIN code image to obtain a VIN code image after the tophat transformation; performing edge detection on the VIN code image after the tophat transformation to obtain a first edge image after edge detection; performing a morphological closing operation on the first edge image to obtain a first closed operation image; performing a morphological opening operation on the first closed operation image to obtain a first open operation image; determining the minimum bounding rectangle of the VIN code area in the first open operation image, and determining the area determined by the minimum bounding rectangle as the VIN code area included in the VIN code image; identifying the VIN code area to identify the VIN code of the vehicle.
[0138] Optionally, the structural element used in the morphological closing operation is a 1*N row vector structural element, where N is the number of columns and is greater than 0; wherein N is set according to the resolution of the vin code image; and / or, N is set according to the size of the vin code area in the vin code image, and the size includes length or width; and / or, N is set according to the ratio of the character spacing in the vin code area to the size of the vin code image; and / or, N is set according to the width of the vin code image.
[0139] Optionally, when the processor 401 processes the license plate image according to a preset license plate image processing rule to obtain the license plate included in the license plate image, it may specifically perform the following steps: performing edge detection on the license plate image to obtain a second edge image after edge detection; performing a morphological closing operation on the second edge image to obtain a second closed operation image; performing a morphological opening operation on the second closed operation image to obtain a second open operation image; determining the minimum circumscribed rectangle of the license plate area in the second open operation image, and determining the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image; and identifying the license plate area to identify the license plate of the vehicle.
[0140] Optionally, the structural element used in the morphological closing operation is an M*N vector structural element, where M is the number of rows, N is the number of columns, and M and N are greater than 0; wherein, M and N are set according to the resolution of the license plate image; and / or, M and N are set according to the size of the license plate area in the license plate image, and the size includes length and / or width; and / or, M and N are set according to the ratio of the character spacing in the license plate area to the size of the license plate image; and / or, M is set according to the length of the license plate image, and N is set according to the width of the license plate image.
[0141] Optionally, before performing the tophat transformation on the VIN code image to obtain the VIN code image after the tophat transformation, the processor 401 may also perform the following steps: detecting whether the brightness of the VIN code image is within a preset brightness range; if the brightness of the VIN code image is not within the brightness range, triggering the step of performing the tophat transformation on the VIN code image to obtain the VIN code image after the tophat transformation.
[0142] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0143] The user interface 403 may include an input device and an output device. The input device may include a touch panel, a microphone, etc., and the output device may include a display (LCD, etc.), a speaker, etc.
[0144] The communication interface 404 may include a receiver and a transmitter for communicating with other devices.
[0145] The memory 402 may include a read-only memory and a random access memory, and provide instructions and data to the processor 401. A portion of the memory 402 may also include a non-volatile random access memory. For example, the memory 402 may also store the above-mentioned correspondence between the function pointer and the function, and the like.
[0146] In a specific implementation, the processor 401 described in the embodiment of the present application may execute the above Figure 1 to Figure 2 The implementation described in the method embodiment shown in the figure can also be implemented in the embodiment of the present application Figure 3 The implementation of each unit described is not described here in detail.
[0147] The present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement Figure 1 to Figure 2 Part or all of the steps in the image processing method described in the corresponding embodiment can also be used to implement the present application. Figure 3 or Figure 4 The functions of the image processing device of the illustrated embodiment are not described in detail here.
[0148] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute part or all of the steps in the above method.
[0149] The computer-readable storage medium may be an internal storage unit of the image processing device described in any of the foregoing embodiments, such as a hard disk or memory of the image processing device. The computer-readable storage medium may also be an external storage device of the image processing device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the image processing device.
[0150] In this application, the term "and / or" is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0151] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0152] The above description is only a partial implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be included in the protection scope of the present application.
Claims
1. An image processing method, characterized in that: include: Acquire a vehicle image, wherein the vehicle image is a VIN code image or a license plate image, wherein the VIN code image is an image including a VIN code area of the vehicle, and the license plate image is an image including a license plate area of the vehicle; the vehicle image is acquired by a photographing device using a target photographing parameter, and the target photographing parameter is determined according to the vehicle model and system time; The vehicle image is identified according to a preset identification rule to identify the type of the vehicle image, and the type of the vehicle image is a VIN code image type or a license plate image type; the preset identification rules include multiple types, and the multiple identification rules at least include a rule for inputting the vehicle image into an image detection model to identify the type of the vehicle image, a rule for taking the type corresponding to the vehicle image with the highest similarity to the vehicle image among the vehicle image samples stored in the database as the type of the vehicle image, and a rule for determining the type of the vehicle image according to the source label of the vehicle image, and the image processing device switches the identification rule according to the preset switching rule to identify the type of the vehicle image; the switching rule includes: a switching rule for switching the identification rule according to a preset time interval, a switching rule for switching the identification rule according to the label of the vehicle image, a switching rule for switching the identification rule according to a switching instruction input by a user, a switching rule for switching the identification rule when the number of errors in the identified VIN code or license plate reaches a preset number, and a switching rule for switching the identification rule when the frequency of errors in the identification of the VIN code or license plate within a preset time period is greater than a preset frequency threshold, and a corresponding relationship between the switching rule and the identification rule is preset; If it is determined that the type of the vehicle image is a vin code image type, the vin code image is processed according to a preset vin code image processing rule to obtain a vin code area included in the vin code image, and the vin code of the vehicle is identified from the vin code area; If it is determined that the type of the vehicle image is a license plate image type, the license plate image is processed according to a preset license plate image processing rule to obtain the license plate area included in the license plate image, and the license plate of the vehicle is identified from the license plate area; the license plate image processing rule and the vin code image processing rule use different area identification methods; the processing corresponding to the vin code image processing rule includes top hat transformation, edge detection, morphological closing operation, morphological opening operation and minimum bounding rectangle recognition, and the structural element used in the morphological closing operation is a 1*N row vector structural element, and N is obtained according to the resolution of the vin code image, the size of the vin code area, the ratio of the character spacing in the vin code area to the size of the vin code image, and the width of the vin code image; the processing corresponding to the license plate image processing rule includes edge detection, morphological closing operation, morphological opening operation and minimum bounding rectangle recognition The structure element used in the morphological closing operation is an M*N vector structure element, and the M and N are obtained according to the resolution of the license plate image, the size of the license plate area, the ratio of the character spacing in the license plate area to the size of the license plate image, and the size of the license plate image, and M is obtained according to the length of the license plate image, and N is obtained according to the width of the license plate image; the edge detection includes convolving the image with a Gaussian filter to smooth the image and filter out noise, calculating the gradient amplitude and direction of each pixel in the image by first-order partial derivative finite difference, performing non-maximum suppression on the gradient amplitude to eliminate the stray response caused by edge detection, and if the gradient value of the edge pixel is higher than a high threshold, it is marked as a strong edge pixel, if the gradient value of the edge pixel is less than a high threshold and greater than a low threshold, it is marked as a weak edge pixel, and if the gradient value of the edge pixel is less than a low threshold, it is suppressed.
2. The method according to claim 1, characterized in that The identifying the vehicle image according to a preset identification rule to identify the type of the vehicle image includes: Inputting the vehicle image into a preset image detection model to obtain an output result of the image detection model, wherein the output result is used to indicate the type of the vehicle image; In which, the image detection model is trained based on multiple pre-selected image samples and the type of each image sample, the multiple image samples include a first number of VIN code image samples and a second number of license plate image samples, and the absolute value of the difference between the first number and the second number does not exceed a preset number threshold.
3. The method according to claim 1 or 2, characterized in that: The VIN code image is processed according to a preset VIN code image processing rule to obtain the VIN code of the vehicle, including: Performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation; Performing edge detection on the VIN code image after the top-hat transformation to obtain a first edge image after edge detection; Performing a morphological closing operation on the first edge image to obtain a first closed operation image; Performing a morphological opening operation on the first closed operation image to obtain a first open operation image; Determine a minimum bounding rectangle of a VIN code area in the first open operation image, and determine the area determined by the minimum bounding rectangle as the VIN code area included in the VIN code image; The VIN code area is identified to identify the VIN code of the vehicle.
4. The method according to claim 3, characterized in that Before performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation, the method further comprises: Detecting whether the brightness of the VIN code image is within a preset brightness range; If the brightness of the VIN code image is not within the brightness interval, the step of performing a top-hat transformation on the VIN code image to obtain a VIN code image after the top-hat transformation is triggered.
5. The method according to claim 1 or 2, characterized in that: The processing of the license plate image according to a preset license plate image processing rule to obtain the license plate of the vehicle includes: Performing edge detection on the license plate image to obtain a second edge image after edge detection; performing a morphological closing operation on the second edge image to obtain a second closed operation image; performing a morphological opening operation on the second closed operation image to obtain a second open operation image; Determine the minimum circumscribed rectangle of the license plate area in the second open operation image, and determine the area determined by the minimum circumscribed rectangle as the license plate area included in the license plate image; The license plate area is identified to identify the license plate of the vehicle.
6. An image processing device, characterized in that: The method comprises means for executing the method according to any one of claims 1 to 5.
7. An image processing device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Vehicle license plate recognition method based on video
CN104050450A
Evidence taking method of vehicle illegal parking and device thereof
CN105894817A
Shooting method and shooting device
CN106558121A
Dynamic video road license plate identification method
CN107067002A
Vehicle loss assessment method and apparatus, electronic device and storage medium
CN108446618A
