A vehicle license plate visual detection method, device, terminal equipment and storage medium

Abstract

The application discloses a vehicle label visual detection method and device, a terminal equipment and a storage medium. The model information of a vehicle to be detected is acquired, and corresponding detection item information is queried according to the model information. A sensing device at a corresponding position is triggered to acquire a to-be-detected image of a corresponding position of the vehicle to be detected according to the detection item information. A corresponding target detection model is selected according to a picture label of the to-be-detected image, label information in the to-be-detected image is detected through the target detection model, to-be-detected sample information of the label information is detected through a pre-trained template matching algorithm, similarity comparison is performed between the to-be-detected sample information and a pre-trained multi-dimensional information sample distribution, a similar field in the label information is corrected, and a target label of the label is obtained. The target label and corresponding probability detected are used as target information of visual detection. The label detection efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle inspection technology, and in particular to a method, apparatus, terminal equipment, and storage medium for visual inspection of vehicle signs. Background Technology

[0002] During the automobile production process, it is usually necessary to affix labels to the vehicle body. Since different vehicle models have different labels, and the placement of the labels on the vehicle body is also different, it is necessary to inspect the labels on each vehicle during the automobile production process to ensure that the labels are affixed correctly.

[0003] Existing vehicle label inspection methods typically involve inspectors on the automobile production line manually checking the accuracy of the label data. However, when manually visually inspecting and verifying vehicle label information, the accuracy and efficiency are both low. This is because labels differ between different models on the production line, and even different versions of the same model may have slight variations. Furthermore, the frequent and uncertain nature of actual work, such as uncontrollable human factors like inspector fatigue, contributes to the problem. Summary of the Invention

[0004] This invention provides a method, apparatus, terminal device, and storage medium for visual inspection of vehicle signs, to solve the problems of low detection efficiency and accuracy of existing visual inspection methods for vehicle signs. By capturing images of vehicle sign targets and performing image processing for recognition and detection, it replaces manual visual inspection and improves detection efficiency and accuracy.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a visual detection method for vehicle signs, comprising the following steps:

[0006] Obtain the model information of the vehicle to be inspected, and query the corresponding inspection item information based on the model information;

[0007] Based on the detection item information, the corresponding sensing device is triggered to acquire the image of the vehicle to be detected at the corresponding location;

[0008] The corresponding target detection model is selected based on the image label of the image to be detected, and the sign information in the image to be detected is detected by the target detection model. The sign information includes sign type, location and probability.

[0009] The test sample information of the sign information is detected by a pre-trained template matching algorithm. The similarity of the test sample information is compared with the pre-trained multidimensional information sample distribution. The similarity fields in the sign information are corrected to obtain the target label of the sign.

[0010] The detected target labels and their corresponding probabilities are used as the target information for visual detection.

[0011] Preferably, the method further includes the following steps:

[0012] The target information and the model information are fed back to the display device for visual display.

[0013] The target information is compared with the standard information pre-stored on the cloud server to determine whether the target information and the standard information are the same;

[0014] When the target information and the standard information are the same, the display device outputs a normal detection feedback and uploads the target information to the cloud server;

[0015] When the target information and the standard information are different, the display device outputs a detection anomaly feedback.

[0016] As a preferred embodiment, before selecting the corresponding target detection model based on the image label of the image to be detected, the method further includes the following steps:

[0017] The AI ​​detection service is activated, and the two-stage Faster RCNN object detection model based on deep learning is used to listen to the HTTP port to determine whether an image input has been detected.

[0018] If no image is input, continue to wait and keep listening on the HTTP port until an image is received.

[0019] If an image is received from the front end, the received image will be used as the image to be detected.

[0020] Preferably, before detecting the sign information in the image to be detected using the target detection model, the method further includes the following steps:

[0021] The image to be detected is corrected using a specular filtering algorithm.

[0022] Preferably, after detecting the sign information in the image to be detected according to the target detection model, the method further includes the following steps:

[0023] The sign information is filtered by using the NMS algorithm to select valid target objects and by using the outlier removal algorithm to remove invalid sign targets.

[0024] As a preferred embodiment, the object detection model is specifically a deep learning-based two-stage Faster R-CNN object detection model.

[0025] Preferably, the template matching algorithm pre-training process includes:

[0026] The sign images in the pre-collected training set are divided into n regions by the Patch Embedding module of the template matching algorithm;

[0027] Each sign image, after being encoded with corresponding locations for different regions, is fed into a Three-stage HPD network to calculate the multidimensional information sample distribution of the sign image. The Three-stage HPD network consists of three identical MixingBlock blocks and a Merging module; the Mixing Block includes Local Mixing and Global Mixing.

[0028] This invention also provides a vehicle sign visual inspection device, comprising:

[0029] The project query module is used to obtain the model information of the vehicle to be inspected and query the corresponding inspection project information based on the model information.

[0030] The image acquisition module is used to trigger the sensing device at the corresponding position to acquire the image to be detected at the corresponding position of the vehicle to be detected based on the detection item information;

[0031] The image detection module is used to select the corresponding target detection model according to the image label of the image to be detected, and to detect the sign information in the image to be detected through the target detection model. The sign information includes sign type, location and probability.

[0032] The target correction module is used to detect the test sample information of the sign information through a pre-trained template matching algorithm, compare the similarity of the test sample information with the pre-trained multidimensional information sample distribution, correct the similarity fields in the sign information, and obtain the target label of the sign.

[0033] The results output module is used to output the detected target labels and their corresponding probabilities as target information for visual detection.

[0034] Preferably, the device further includes a result determination module, used for:

[0035] The target information and the model information are fed back to the display device for visual display.

[0036] The target information is compared with the standard information pre-stored on the cloud server to determine whether the target information and the standard information are the same;

[0037] When the target information and the standard information are the same, the display device outputs a normal detection feedback and uploads the target information to the cloud server;

[0038] When the target information and the standard information are different, the display device outputs a detection anomaly feedback.

[0039] As a preferred embodiment, the device further includes a listening module for:

[0040] Before selecting the corresponding target detection model based on the image label of the image to be detected, the AI ​​detection service is started. The two-stage Faster RCNN target detection model based on deep learning listens to the HTTP port to determine whether the image input is detected.

[0041] If no image is input, continue to wait and keep listening on the HTTP port until an image is received.

[0042] If an image is received from the front end, the received image will be used as the image to be detected.

[0043] As a preferred embodiment, the device further includes a calibration module for:

[0044] Before detecting the sign information in the image to be detected by the target detection model, the image to be detected is corrected by a specular filtering algorithm.

[0045] As a preferred embodiment, the device further includes a screening module for:

[0046] After detecting the sign information in the image to be detected according to the target detection model, the NMS algorithm is used to filter the valid target objects in the sign information, and the outlier removal algorithm is used to remove invalid sign targets, thus filtering the sign information.

[0047] Preferably, the target detection model is a deep learning-based two-stage Faster RCNN target detection model.

[0048] As a preferred embodiment, the process of the target correction module training the template matching algorithm includes:

[0049] The sign images in the pre-collected training set are divided into n regions by the Patch Embedding module of the template matching algorithm;

[0050] Each sign image, after being encoded with corresponding locations for different regions, is fed into a Three-stage HPD network to calculate the multidimensional information sample distribution of the sign image. The Three-stage HPD network consists of three identical MixingBlock blocks and a Merging module; the Mixing Block includes Local Mixing and Global Mixing.

[0051] This invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the vehicle sign visual detection method as described in any of the first aspects.

[0052] This invention provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the vehicle sign visual detection method as described in any of the first aspects.

[0053] Compared with the prior art, the beneficial effect of the embodiments of the present invention is that by taking pictures of the vehicle sign targets and using image processing to recognize and detect the pictures, it replaces manual visual inspection, thereby solving the problem of low detection efficiency and accuracy of existing vehicle sign visual inspection methods and improving detection efficiency and accuracy. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a visual inspection method for vehicle signs provided in an embodiment of the present invention;

[0055] Figure 2 This is a flowchart illustrating a vehicle sign visual inspection method according to another embodiment of the present invention;

[0056] Figure 3 This is a flowchart illustrating a vehicle sign visual inspection method according to another embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the Global Mixing structure provided in an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram of the Local Mixing structure provided in an embodiment of the present invention;

[0059] Figure 6 This is a schematic diagram of the structure of the vehicle sign visual inspection device in an embodiment of the present invention;

[0060] Figure 7This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] In the embodiments provided by the present invention, see Figure 1 This is a flowchart illustrating a vehicle sign visual inspection method according to an embodiment of the present invention, including steps S1 to S5:

[0063] S1, Obtain the model information of the vehicle to be tested, and query the corresponding test item information based on the model information;

[0064] S2, based on the detection item information, trigger the corresponding sensing device to acquire the image to be detected at the corresponding position of the vehicle to be detected;

[0065] S3, select the corresponding target detection model according to the image label of the image to be detected, and detect the sign information in the image to be detected by the target detection model. The sign information includes sign type, location and probability.

[0066] S4, detect the test sample information of the sign information through a pre-trained template matching algorithm, compare the similarity of the test sample information with the pre-trained multidimensional information sample distribution, correct the similarity fields in the sign information, and obtain the target label of the sign;

[0067] S5 uses the detected target label and its corresponding probability as the target information for visual detection.

[0068] In the specific implementation of this embodiment, when performing visual inspection of vehicle signs, it is necessary to first obtain the model information of the vehicle to be inspected. Different vehicle models correspond to different inspection items. Therefore, it is necessary to be able to query the corresponding inspection item information based on the model information of the vehicle to be inspected. The inspection item information is the number of signs to be inspected for each vehicle and the distribution position of the signs on the vehicle to be inspected. By determining the inspection item information, the corresponding sign image can be obtained for corresponding visual inspection processing.

[0069] When visual inspection is performed on the vehicle to be inspected, corresponding sensing devices are pre-configured around the location of the vehicle to be inspected, and the corresponding sign images are acquired through the sensing devices.

[0070] It should be noted that when performing visual inspection of vehicle signs, several sensing devices are first arranged at corresponding positions on the inspection rack. When inspecting the signs, the vehicle to be inspected is placed in a preset position, and the sensing devices arranged at this time can acquire images of the corresponding positions.

[0071] Based on the detection item information, the corresponding sensing device is triggered to acquire the image of the vehicle to be detected at the corresponding location;

[0072] The corresponding target detection model is selected based on the image label of the image to be detected, and the sign information in the image to be detected is detected by the target detection model. The sign information includes the corresponding sign type, sign location and sign detection probability.

[0073] The target labels of similar signs are corrected by using a pre-trained template matching algorithm. The template matching algorithm analyzes the multidimensional information sample distribution of the detection items from historical image data, and then compares the similarity between the information of the sample to be tested and the multidimensional information sample distribution. Based on the image similarity, it is determined whether the detected target is correct, so as to correct the similarity field in the sign information, obtain the target label of the sign, and reduce the false detection rate of sign detection.

[0074] The detected target labels and their corresponding probabilities are combined into JSON and returned to the front end as target information for visual detection; this completes the sign detection of the vehicle to be detected, enabling efficient and accurate sign detection.

[0075] This embodiment can solve the problems of low detection efficiency and accuracy of existing vehicle sign visual inspection methods. By taking pictures of vehicle sign targets and performing image processing to recognize and detect the pictures, it replaces manual visual inspection and improves detection efficiency and accuracy.

[0076] In another embodiment of the present invention, the method further includes the following steps:

[0077] The target information and the model information are fed back to the display device for visual display.

[0078] The target information is compared with the standard information pre-stored on the cloud server to determine whether the target information and the standard information are the same;

[0079] When the target information and the standard information are the same, the display device outputs a normal detection feedback and uploads the target information to the cloud server;

[0080] When the target information and the standard information are different, the display device outputs a detection anomaly feedback.

[0081] In the specific implementation of this embodiment, please refer to Figure 2 This is a flowchart illustrating a vehicle sign visual inspection method according to another embodiment of the present invention.

[0082] When performing visual inspection of vehicle signs, after the vehicle arrives, the vehicle identification number (VIN) is entered by scanning the barcode on the side of the vehicle. The system recognizes the VIN, then calls the AS400 interface to obtain vehicle information, including the vehicle model. Based on the vehicle model, the system obtains the vehicle inspection items, triggers the inspection, pushes messages to the large screen, and sends voice prompts. The large screen message push prompts the quality inspector to operate according to the steps.

[0083] After the barcode scanner is triggered, the driver will operate according to the instructions on the large screen and voice prompts to retrieve an image from the camera.

[0084] Push notifications, push messages to the large screen, send voice notifications, and push messages through the large screen;

[0085] The camera captures an image, the model retrieves the image, and the AI ​​intelligent detection service detects the image and outputs the detection results.

[0086] The test results are summarized and pushed out via large screen message push, and the target information and model information are fed back to the display device for visual display;

[0087] The target information is compared with the standard information pre-stored on the cloud server to determine whether the target information and the standard information are the same, i.e., OK / NG judgment is performed;

[0088] When the target information and the standard information are different, the output result is NG, the display device outputs abnormal feedback, and a voice prompt is pushed. Triggering the alarm light and voice prompt will trigger a red indicator on the large screen, alerting on-site personnel that there is a problem with the vehicle.

[0089] When the target information and the standard information are the same, the output result is OK, the display device outputs a normal detection feedback, a green mark appears on the large screen, and the target information is sent to the designated server, that is, the target information is uploaded to the cloud server for manual confirmation.

[0090] When the target information is manually confirmed to be abnormal, the detection results are modified, the vehicle information is modified, and after the modification is completed, the modified target information is uploaded to the designated server again.

[0091] If the target information is manually confirmed to be normal, no modifications are made.

[0092] After receiving the vehicle test results on the large screen, on-site personnel can modify the test results according to the actual situation on site. The vehicle test data and image data are stored in the corresponding server and management platform, which can be queried and traced in real time.

[0093] It can automatically detect signs and verify information, and display and alarm for incorrect signs;

[0094] Visual inspection of signs can be combined with other intelligent inspection projects for simultaneous operation, reducing the number of positions and optimizing the process; it enables automatic detection and recording of sign inspection information, ensuring accurate recording of all vehicles; the sign inspection results are automatically saved and uploaded to a designated server, and the final results are displayed as images and data.

[0095] In another embodiment of the present invention, before selecting the corresponding target detection model based on the image label of the image to be detected, the method further includes the following steps:

[0096] The AI ​​detection service is activated, and the two-stage Faster RCNN object detection model based on deep learning is used to listen to the HTTP port to determine whether an image input has been detected.

[0097] If no image is input, continue to wait and keep listening on the HTTP port until an image is received.

[0098] If an image is received from the front end, the received image will be used as the image to be detected.

[0099] In the specific implementation of this embodiment, please refer to Figure 3 This is a flowchart illustrating a vehicle sign visual inspection method according to another embodiment of the present invention.

[0100] Before selecting the corresponding target detection model based on the image label of the image to be detected, the AI ​​detection service is started first, and the N detection services of the sign AI service array listen to the corresponding HTTP ports.

[0101] To determine whether the front-end load is balanced when sending images, a two-stage Faster R-CNN object detection model based on deep learning is used to listen to the HTTP port and determine whether image input is detected.

[0102] If no image is input, continue to wait and keep listening on the HTTP port until an image is received.

[0103] If an image is received from the front end, the received image is used as the image to be detected;

[0104] By using AI detection services to inspect HTTP interfaces, we can ensure the high efficiency of AI-powered intelligent detection services.

[0105] In another embodiment of the present invention, before detecting the sign information in the image to be detected by the target detection model, the method further includes the following steps:

[0106] The image to be detected is corrected using a specular filtering algorithm.

[0107] In the specific implementation of this embodiment, please refer to Figure 3 After selecting the detection model for sign detection by image tags, that is, after selecting the corresponding target detection model according to the image tags of the image to be detected, the image quality is corrected and the false negative rate is reduced by image highlight filtering algorithm.

[0108] Before conducting sign inspection, a highlight filtering algorithm is used to correct image quality, reduce the false negative rate, and improve the accuracy of visual sign inspection.

[0109] In another embodiment of the present invention, after detecting the sign information in the image to be detected according to the target detection model, the method further includes the following steps:

[0110] The sign information is filtered by using the NMS algorithm to select valid target objects and by using the outlier removal algorithm to remove invalid sign targets.

[0111] In the specific implementation of this embodiment, please refer to Figure 3 After detecting the image to be detected according to the target detection model, that is, after the target detection model detects the sign information such as the type, location and probability of the sign in the image, the overlapping detection boxes are filtered out by the NMS algorithm, that is, the valid detection targets in the sign information are filtered out by the NMS algorithm, and the invalid location sign targets are filtered out by the outlier filtering algorithm, thus filtering the sign information.

[0112] The NMS algorithm is used to filter out valid detection targets, and the outlier filtering algorithm is used to filter out invalid location signs, thereby reducing the false detection rate.

[0113] In another embodiment of the present invention, the object detection model is specifically a deep learning-based two-stage Faster RCNN object detection model.

[0114] In this specific implementation, the target detection model is a deep learning-based two-stage Faster RCNN target detection model;

[0115] The two-stage Faster R-CNN object detection model based on deep learning can accurately detect sign targets using detection boxes of appropriate size.

[0116] In another embodiment provided by the present invention, the template matching algorithm pre-training process includes:

[0117] The sign images in the pre-collected training set are divided into n regions by the Patch Embedding module of the template matching algorithm;

[0118] Each sign image, after being encoded with corresponding locations for different regions, is fed into a Three-stage HPD network to calculate the multidimensional information sample distribution of the sign image. The Three-stage HPD network consists of three identical MixingBlock blocks and a Merging module; the Mixing Block includes Local Mixing and Global Mixing.

[0119] In this specific implementation, during the template matching algorithm training stage, the sign images in the pre-collected training set are first divided into n regions by the Patch Embedding module. After the corresponding position encoding of different regions is performed, they are sent to the Three-stage HPD network to calculate the multidimensional information sample distribution of the sign images.

[0120] The Three-Stage HDP network consists of three identical Mixing Blocks and Merging modules. The Mixing Block is divided into Local Mixing and Global Mixing.

[0121] See Figure 4 This is a schematic diagram of the Global Mixing structure provided in this embodiment of the invention. Global Mixing is used to evaluate the dependencies between all character components. When entering a mixing block, a layer norm is applied, and then a multi-head self-attention mechanism is applied for dependency modeling. The layer norm and multilayer perceptron are applied sequentially for feature fusion, which, together with shortcut connections, forms Global Mixing.

[0122] See Figure 5 This is a schematic diagram of the Local Mixing structure provided in this embodiment of the invention. Local Mixing evaluates the correlation between components within a predefined window. Its purpose is to encode morphological features and establish the correlation between components within the features, thereby simulating stroke-like features that are crucial for feature recognition. Then, the obtained multidimensional features and sign semantic features are used to calculate the loss.

[0123] After training, a pre-trained Three-Stage HPD network is obtained. The sign image is mapped onto a Gaussian distribution matrix after passing through the pre-trained model. During the testing phase, the sign image passes through the Patch Embedding module and the Position Embedding module, and then the Three-Stage HDP network extracts features before feeding them into the Gaussian distribution matrix. This determines whether the distribution area is a target detection area, thereby reducing the probability of false detections. Finally, the labels and probabilities of the effectively detected targets are combined into JSON and returned to the front end. The front end then displays the visualization results by combining the detection information with the model information.

[0124] This invention uses GPUs for deep learning model computation and implements various control functions through programming. It features a high degree of automation and intelligence, and can automatically check and verify the logo information of various vehicle models. It uses deep learning to recognize the logos of different vehicle models and is compatible with multiple existing models. When a new vehicle model is produced, images of the new model are taken for model training. After the model is trained, it is released to replace the old model, thus realizing the inspection of new vehicle models. The subsequent maintenance is simple. The developed cloud server allows for convenient comparison, querying, and traceability of data.

[0125] In yet another embodiment provided by the present invention, see Figure 6 This is a schematic diagram of the structure of a vehicle sign visual inspection device provided in an embodiment of the present invention. The device includes:

[0126] The project query module is used to obtain the model information of the vehicle to be inspected and query the corresponding inspection project information based on the model information.

[0127] The image acquisition module is used to trigger the sensing device at the corresponding position to acquire the image to be detected at the corresponding position of the vehicle to be detected based on the detection item information;

[0128] The image detection module is used to select the corresponding target detection model according to the image label of the image to be detected, and to detect the sign information in the image to be detected through the target detection model. The sign information includes sign type, location and probability.

[0129] The target correction module is used to detect the test sample information of the sign information through a pre-trained template matching algorithm, compare the similarity of the test sample information with the pre-trained multidimensional information sample distribution, correct the similarity fields in the sign information, and obtain the target label of the sign.

[0130] The results output module is used to output the detected target labels and their corresponding probabilities as target information for visual detection.

[0131] It should be noted that the vehicle sign visual inspection device provided in this embodiment of the invention can realize all the processes of the vehicle sign visual inspection method described in any of the above embodiments. The functions and technical effects of each module in the device are the same as the functions and technical effects of the vehicle sign visual inspection method described in the above embodiments, and will not be repeated here.

[0132] See Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the vehicle sign visual detection method as described in any embodiment of the first aspect.

[0133] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. The terminal device may also include input / output devices, network access devices, buses, etc.

[0134] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0135] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0136] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the vehicle sign visual detection method as described in any embodiment of the first aspect.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary hardware platforms, and of course, it can also be implemented entirely by hardware. Based on this understanding, all or part of the technical solution of the present invention that contributes to the background art can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0138] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A visual inspection method for vehicle signs, characterized in that, Includes the following steps: Obtain the model information of the vehicle to be inspected, and query the corresponding inspection item information based on the model information; Based on the detection item information, the corresponding sensing device is triggered to acquire the image of the vehicle to be detected at the corresponding location. The corresponding target detection model is selected based on the image label of the image to be detected. The image to be detected is corrected by a specular filtering algorithm. The sign information in the corrected image to be detected is detected by the target detection model. The sign information includes sign type, location and probability. The sign information is filtered by using the NMS algorithm to select valid target objects and by using the outlier removal algorithm to remove invalid sign targets. The sign information is then filtered by using a pre-trained template matching algorithm to detect the test sample information of the filtered sign information. The test sample information is then compared with the distribution of multidimensional information samples obtained by pre-training to correct the similarity fields in the sign information and obtain the target label of the sign. The detected target labels and their corresponding probabilities are used as the target information for visual detection.

2. The vehicle sign visual inspection method as described in claim 1, characterized in that, The method further includes the following steps: The target information and the model information are fed back to the display device for visual display. The target information is compared with the standard information pre-stored on the cloud server to determine whether the target information and the standard information are the same; When the target information and the standard information are the same, the display device outputs a normal detection feedback and uploads the target information to the cloud server; When the target information and the standard information are different, the display device outputs a detection anomaly feedback.

3. The vehicle sign visual inspection method as described in claim 1, characterized in that, Before selecting the corresponding target detection model based on the image label of the image to be detected, the method further includes the following steps: The AI ​​detection service is activated, and the two-stage Faster RCNN object detection model based on deep learning is used to listen to the HTTP port to determine whether an image input has been detected. If no image is input, continue to wait and keep listening on the HTTP port until an image is received. If an image is received from the front end, the received image will be used as the image to be detected.

4. The vehicle sign visual inspection method as described in claim 1, characterized in that, The object detection model is specifically a two-stage Faster RCNN object detection model based on deep learning.

5. The vehicle sign visual inspection method as described in claim 1, characterized in that, The template matching algorithm pre-training process includes: The sign images in the pre-collected training set are divided into n regions by the Patch Embedding module of the template matching algorithm; Each sign image, after being encoded with corresponding location information for different regions, is fed into a Three-stage HPD network to calculate the multidimensional information sample distribution of the sign image. The Three-stage HPD network consists of three identical MixingBlock blocks and a Merging module; the Mixing Block includes Local Mixing and Global Mixing.

6. A vehicle sign visual inspection device, characterized in that, include: The project query module is used to obtain the model information of the vehicle to be inspected and query the corresponding inspection project information based on the model information. The image acquisition module is used to trigger the sensing device at the corresponding position to acquire the image to be detected at the corresponding position of the vehicle to be detected based on the detection item information; The image detection module is used to select the corresponding target detection model according to the image label of the image to be detected, correct the image to be detected by the highlight filtering algorithm, and detect the sign information in the corrected image to be detected by the target detection model. The sign information includes sign type, location and probability. The target correction module is used to filter valid detection targets in the sign information using the NMS algorithm and filter invalid location sign targets using the outlier removal algorithm. The sign information is filtered, and the test sample information of the filtered sign information is detected by the pre-trained template matching algorithm. The test sample information is compared with the similarity of the pre-trained multidimensional information sample distribution to correct the similarity fields in the sign information and obtain the target label of the sign. The results output module is used to output the detected target labels and their corresponding probabilities as target information for visual detection.

7. A terminal device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the vehicle sign visual inspection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the vehicle sign visual inspection method as described in any one of claims 1 to 5.

Images