Vehicle detection system and method and computer equipment

Abstract

The invention relates to a vehicle detection system, method and device, computer equipment, a storage medium and a computer program product. The system comprises a transmission device configured to transmit a to-be-detected vehicle to a vehicle detection station, an image acquisition device configured to acquire a vehicle image of the to-be-detected vehicle at the vehicle detection station, and a control device configured to respond to a detection scene selection event, and to determine whether the to-be-detected vehicle is in a plurality of candidate detection scenes. And determining a target scene configured for the to-be-detected vehicle. And the control equipment is also used for determining a detection item in the target scene based on the vehicle type to which the to-be-detected vehicle belongs, and further analyzing the vehicle image according to the detection item to obtain a detection result. By adopting the system, the vehicle type of the to-be-detected vehicle and the personalized detection requirement for the to-be-detected vehicle are considered in the determination of the detection item, so that the accuracy of the detection result obtained by analyzing the vehicle image according to the detection item is relatively high.

Description

Technical Field

[0001] The present application relates to the field of vehicle detection technology, and in particular to a vehicle detection system, method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the vehicle production line, in order to ensure the quality of vehicle products, it is necessary to conduct quality inspection on the whole vehicle. In the traditional technology, quality inspectors conduct quality inspection on each area of ​​the car, which has the problem of low work efficiency.

[0003] With the development of technology, automatic detection technology based on neural network models has emerged. By using pre-trained visual detection models to detect vehicles, the efficiency of vehicle detection can be improved to a certain extent. However, the detection objects are limited by the labeled content of the training samples, and there is a problem of low accuracy of the detection results. Summary of the invention

[0004] Based on this, it is necessary to provide a vehicle detection system, method, device, computer equipment, computer-readable storage medium and computer program product that can detect the accuracy of the results in response to the above technical problems.

[0005] In a first aspect, the present application provides a vehicle detection system, the system comprising:

[0006] A transmission device is used to transmit the vehicle to be tested to a vehicle inspection station; an image acquisition device is used to acquire a vehicle image of the vehicle to be tested at the vehicle inspection station; a control device is used to: in response to a detection scene selection event, determine a target scene configured for the vehicle to be tested from multiple candidate detection scenes; based on the model of the vehicle to be tested, determine the detection items under the target scene; analyze the vehicle image according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0007] In a second aspect, the present application also provides a vehicle detection method, which is applied to a control device described in any one of the above systems, and the method includes: in response to a detection scene selection event, determining a target scene configured for the vehicle to be tested from multiple candidate detection scenes; based on the model of the vehicle to be tested, determining the detection items under the target scene; analyzing the vehicle image according to the detection items, and determining the detection result of the vehicle to be tested corresponding to the target scene.

[0008] In a third aspect, the present application also provides a vehicle detection device. The device comprises:

[0009] A scene determination module is used to determine a target scene configured for the vehicle to be tested from multiple candidate detection scenes in response to a detection scene selection event; a detection item determination module is used to determine the detection items under the target scene based on the model of the vehicle to be tested; and an image analysis module is used to analyze the vehicle image according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0010] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0011] In response to a detection scene selection event, a target scene configured for the vehicle to be tested is determined from multiple candidate detection scenes; based on the model of the vehicle to be tested, the detection items under the target scene are determined; and the vehicle image is analyzed according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0012] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0013] In response to a detection scene selection event, a target scene configured for the vehicle to be tested is determined from multiple candidate detection scenes; based on the model of the vehicle to be tested, the detection items under the target scene are determined; and the vehicle image is analyzed according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0014] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0015] In response to a detection scene selection event, a target scene configured for the vehicle to be tested is determined from multiple candidate detection scenes; based on the model of the vehicle to be tested, the detection items under the target scene are determined; and the vehicle image is analyzed according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0016] The above-mentioned vehicle detection system, method, device, computer equipment, storage medium and computer program product are configured with a transmission device for transmitting the vehicle to be tested to the vehicle detection station, an image acquisition device for collecting the vehicle image of the vehicle to be tested at the vehicle detection station, and a control device for responding to the detection scene selection event, and determining the target scene configured for the vehicle to be tested from multiple candidate detection scenes, which can configure a personalized target scene for the vehicle to be tested, and provide a basis for the subsequent determination of the detection items. Furthermore, the control device is also used to determine the detection items under the target scene based on the model of the vehicle to be tested, and further analyze the vehicle image according to the detection items to obtain the detection results. With the above-mentioned system, since the determination of the detection items takes into account the model of the vehicle to be tested and the personalized detection requirements for the vehicle to be tested, the accuracy of the detection results obtained by analyzing the vehicle image according to the detection items is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG1 is a schematic diagram of the structure of a vehicle detection system in one embodiment;

[0018] FIG2 is a schematic flow chart of a method for determining a detection result of a target scene in one embodiment;

[0019] FIG3 is a schematic diagram of a vehicle test area in one embodiment;

[0020] FIG4 is a visualization diagram of determining text detection results in one embodiment;

[0021] FIG5 is a schematic diagram of a process flow of a vehicle detection method in one embodiment;

[0022] FIG6 is a schematic diagram of a flow chart of a method for visually inspecting a vehicle body in an embodiment;

[0023] FIG. 7 is a schematic diagram of a full vehicle body anti-foolproofing scenario item in one embodiment;

[0024] FIG8 is a schematic diagram of a quality inspection page in one embodiment;

[0025] FIG9 is a schematic diagram of the structure of a vehicle detection device in one embodiment;

[0026] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] In one embodiment, as shown in FIG. 1 , the embodiment of the present application provides a vehicle detection system, including: a transmission device 110 , an image acquisition device 120 and a control device 130 .

[0029] The conveying device 110 is used to transfer the vehicle to be tested to the vehicle testing station.

[0030] The vehicle to be tested may be any vehicle in a vehicle production line that needs to be tested.

[0031] The conveying device may be a device for conveying the vehicle to be tested. For example, the conveying device may be a conveying tray or a conveyor belt for carrying the vehicle to be tested. The conveying device may be fixedly connected to the vehicle to be tested, and the conveying device may also support the vehicle to be tested. The connection relationship between the conveying device and the vehicle to be tested is not limited here.

[0032] The vehicle inspection station can be a station in the vehicle production line for inspecting vehicles. A variety of inspection-related equipment is arranged around the vehicle inspection station. For example, a visual camera, a grayscale camera, a proximity switch, etc. A variety of stations are also included in the vehicle assembly line. For example, the vehicle assembly line can include a vehicle inspection station, a manual inspection station, a vehicle splicing station, a vehicle buffer station, etc.

[0033] Specifically, the conveying device is used to transfer the vehicle to be tested to the vehicle testing station.

[0034] In one embodiment, the transmission device may further send a vehicle arrival signal when the vehicle to be tested arrives at the vehicle inspection station. The transmission device may send the vehicle arrival signal to the control device.

[0035] In one embodiment, the conveying device may also send an abnormality arrival signal when the vehicle to be tested has not arrived at the vehicle inspection station. The conveying device may send the abnormality arrival signal to the control device. The abnormality arrival signal may be used to indicate that the vehicle to be tested has not arrived at the vehicle inspection station or that the conveying device itself is abnormal.

[0036] The image acquisition device 120 is used to acquire a vehicle image of the vehicle to be tested at the vehicle testing station.

[0037] The image acquisition device is connected to the control device, and the image acquisition device can acquire images toward the vehicle to be tested.

[0038] An image acquisition device can be a device with image acquisition capabilities. The main task of an image acquisition device is to optically image the characteristic information of a target object (such as a vehicle or other object), convert the optical signal into an electrical signal, and then transmit it to a computer or image processing system. For example, an image acquisition device can be an industrial camera, a scanner, a visual camera, a CCD camera, etc.

[0039] The vehicle image represents an image obtained by processing the electrical signal corresponding to the optical characteristic information of the vehicle to be tested. The vehicle image can be a whole vehicle image or a partial vehicle image.

[0040] Specifically, the image acquisition device can acquire the vehicle image of the vehicle to be tested in real time at the vehicle detection station.

[0041] In one of the embodiments, the image acquisition device may acquire the vehicle image of the vehicle to be tested according to a preset acquisition cycle.

[0042] In one embodiment, the image acquisition device can acquire multiple images of the vehicle to be tested. The image acquisition device can also filter out vehicle images that meet image filtering conditions from each captured image. For example, images with non-horizontal image acquisition angles are filtered out.

[0043] The control device 130 is used to determine a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes in response to a detection scene selection event.

[0044] Among them, the control device can be connected to the image acquisition device and the transmission device respectively.

[0045] The control device may be a device with image processing capabilities. The control device may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc.

[0046] The detection scene selection event may be a selection event triggered manually or automatically on the control device.

[0047] The candidate detection scene may represent the detection scene of the vehicle to be tested, and may be at least one of the following scenes: appearance detection scene, color detection scene, tire detection scene, text detection scene, etc.

[0048] The target scene may represent a candidate detection scene for the vehicle configuration determined among the candidate detection scenes, for example, a color detection scene is selected from the appearance detection scene and the color detection scene.

[0049] Specifically, the control device may determine a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes in response to a detection scene selection event.

[0050] In one embodiment, the control device may further determine and select at least two detection scenarios from a plurality of candidate detection scenarios. In response to the detection scenario selection event, the control device may determine a target scenario configured for the vehicle to be tested from the detection scenarios.

[0051] In one embodiment, the control device obtains environmental information of the detection station. The control device can determine at least two detection scenes that match the environmental information from multiple candidate detection scenes. For example, the environmental information can be that the ambient light intensity meets the ambient light intensity condition, and the color and text detection accuracy is higher at this time.

[0052] The control device 130 is further configured to determine the detection items in the target scene based on the type of vehicle to be tested.

[0053] The vehicle model of the vehicle to be tested may be a model used to identify the vehicle to be tested.

[0054] The inspection items may be specific items included in each candidate inspection scene. For example, for an appearance inspection scene, the specific items in the appearance inspection scene may include a vehicle body, a roof, a hood, a front bumper upper cover, front and rear bumpers, and the like.

[0055] Specifically, the control device may determine the detection items in the target scene based on the type of vehicle to be tested.

[0056] In one embodiment, the control device can obtain multiple recommended items configured for the vehicle model and target scene of the vehicle to be tested. Among them, the recommended items can be items commonly used for detecting the vehicle model in the target scene for the vehicle model. The control device can select target recommended items that meet the item screening conditions from each recommended item, and the target recommended items are detection items in the target scene. For example, 5 recommended items can be randomly selected from 10 recommended items as detection items in the target scene.

[0057] In one embodiment, the control device may obtain multiple candidate items configured for the vehicle model to which the vehicle to be tested belongs. The control device may determine at least two selected items matching the target scene from each candidate item. The control device may determine the detection item under the target scene in response to the detection item determination event. The detection item includes at least a portion of each selected item.

[0058] The control device 130 is further configured to analyze the vehicle image according to the detection items and determine the detection result of the vehicle to be detected corresponding to the target scene.

[0059] The detection result may be an analysis result obtained by analyzing the vehicle image.

[0060] Specifically, the control device analyzes the vehicle image according to the detection items to obtain the detection results of the vehicle to be tested corresponding to the detection items. In the case of multiple detection items, the control device can combine the detection results of each detection item to obtain the detection results of the vehicle to be tested corresponding to the target scene.

[0061] In one embodiment, the target scene may contain only one detection item, and the control device may analyze the vehicle image according to a plurality of image analysis methods matched with the detection items to obtain the detection result of the vehicle to be tested corresponding to the target scene. The image analysis method may be at least one of image preprocessing, image feature extraction, and image segmentation. Image preprocessing includes image transformation and image enhancement. Image transformation includes transformation in the spatial domain, transformation in the spatial domain and frequency domain, and transformation in the frequency domain. Image enhancement includes point operation algorithms and domain algorithms. Point operation algorithms include linear grayscale transformation, nonlinear grayscale transformation, histogram transformation, etc.; domain algorithms include smoothing, sharpening, etc.

[0062] In one embodiment, the target scene may include multiple detection items, and the control device may perform image segmentation on the vehicle image according to the vehicle area to be tested indicated by the detection item, and determine the regional image of the vehicle area to be tested from the vehicle image. The control device may perform image detection processing on the regional image according to the detection content of the vehicle area to be tested in the detection item, and determine the project detection results of the vehicle to be tested corresponding to the detection item. The control device may determine the detection results of the vehicle to be tested corresponding to the target scene in combination with the project detection results of each detection item.

[0063] In one of the embodiments, the vehicle image may be a whole vehicle image, and the whole vehicle image may be a global image representing the vehicle to be tested. Specifically, the image acquisition device may be used to acquire the whole vehicle image of the vehicle to be tested at the vehicle detection station. On this basis, the control device may determine the target scene configured for the vehicle to be tested from a plurality of candidate detection scenes in response to a detection scene selection event. The control device may determine the detection items under the target scene based on the vehicle model to which the vehicle to be tested belongs. The control device may analyze the whole vehicle image according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

[0064] In one embodiment, the vehicle image may be a partial image, wherein the partial image may be an image taken at different positions of the vehicle to be tested, for example, a tire image, a wheel image, a rear image, a side image, etc.

[0065] Specifically, the image acquisition device can be used to collect multiple partial images of the vehicle to be tested at the vehicle inspection station. On this basis, the control device can determine the target scene configured for the vehicle to be tested from multiple candidate detection scenes in response to the detection scene selection event. The control device can determine the target partial image that matches the target scene in each partial image. The control device can determine the detection items under the target scene based on the model of the vehicle to be tested. The control device can analyze the target partial image according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene. Among them, the target scene can be a tire detection scene, and the target partial image that matches the tire detection scene can be a tire image.

[0066] In the above-mentioned vehicle detection system, a transmission device is configured to transmit the vehicle to be tested to the vehicle detection station, an image acquisition device is configured to collect the vehicle image of the vehicle to be tested at the vehicle detection station, and a control device is configured to respond to the detection scene selection event, and determine the target scene configured for the vehicle to be tested from multiple candidate detection scenes, so that a personalized target scene can be configured for the vehicle to be tested, providing a basis for the subsequent determination of the detection items. Furthermore, the control device is also used to determine the detection items under the target scene based on the model of the vehicle to be tested, and further analyze the vehicle image according to the detection items to obtain the detection results. With the above-mentioned system, since the determination of the detection items takes into account the model of the vehicle to be tested and the personalized detection requirements for the vehicle to be tested, the detection results obtained by analyzing the vehicle image according to the detection items are more accurate.

[0067] In one embodiment, when the control device determines the detection items under the target scene based on the model of the vehicle to be tested, it is configured to: obtain multiple candidate items configured for the model of the vehicle to be tested, determine at least two selected items matching the target scene from each candidate item, and determine the detection items under the target scene in response to a detection item determination event, the detection items including at least a part of each selected item.

[0068] The candidate items may be pre-configured specific items, for example, the candidate items include the vehicle body, the roof, the engine cover, the front bumper cover, the front and rear bumpers, the tail logo, the spoiler, the trunk cover, the front and rear bumpers, the left and right exterior mirrors, etc.

[0069] The detection item determination event may be a determination event manually or automatically triggered on the control device, and the determination event is used to determine the detection item in the target scene.

[0070] It should be noted that the detection items include at least a portion of each selected item. In other words, the detection items may include at least a portion of the selected items, and the detection items may also include other items in each candidate item except the selected item.

[0071] Specifically, the control device may obtain in advance a mapping relationship between candidate detection scenes and candidate items. The control device may determine at least two selected items matching the target scene from the candidate items according to the mapping relationship.

[0072] In one embodiment, the detection item determination event may include an item expansion event. Specifically, the control device may screen the expansion items from the candidate items other than the selected items in response to the item expansion event. The control device determines the expansion items and the selected items as the detection items in the target scenario.

[0073] In one embodiment, the detection item determination event may include a project customization event. Specifically, the control device may determine the customized item under the target scene in response to the detection item determination event. The control device may determine the customized item and each selected item as the detection item under the target scene. The customized item may be a new candidate item configured for the target scene.

[0074] In one embodiment, the detection items may include both item expansion events and item customization events. Specifically, the control device may screen the expansion items from candidate items other than the selected items in response to the item expansion events. The control device may determine the customization items under the target scenario in response to the item customization events. The control device may determine the expansion items and the customization items as the detection items under the target scenario.

[0075] In this embodiment, multiple candidate items are obtained for the model configuration of the vehicle to be tested, and at least two selected items that match the target scene are determined from each candidate item. In response to a detection item determination event, the detection items under the target scene are determined. The detection items include at least a part of each selected item. The candidate items are screened multiple times to obtain accurate detection items under the target scene, which is beneficial to improving the accuracy of subsequent vehicle detection.

[0076] In one embodiment, as shown in FIG2 , the number of detection items is multiple;

[0077] When the control device analyzes the vehicle image according to the detection items and determines the detection result of the vehicle to be tested corresponding to the target scene, it is configured as S202 to S206, including:

[0078] S202 , for each detection item, performing image segmentation on the vehicle image according to the vehicle to-be-detected area indicated by the detection item, and determining a regional image of the vehicle to-be-detected area from the vehicle image.

[0079] The vehicle test area may represent an area corresponding to the detection item in the vehicle image. Taking the detection item as a side skirt guard plate as an example, the area where the side skirt guard plate is located in the test image represents the vehicle test area.

[0080] The regional image may be an image corresponding to the region obtained after segmenting the vehicle image.

[0081] Specifically, the control device may determine the vehicle to-be-tested area indicated by the detection item in the vehicle image according to the standard image features corresponding to the detection item. The vehicle to-be-tested area may be an identification frame corresponding to the detection item.

[0082] Specifically, after obtaining the vehicle to-be-tested area, the control terminal may perform image segmentation on the vehicle image according to the position information of the recognition frame, and determine the regional image of the vehicle to-be-tested area from the vehicle image.

[0083] In one embodiment, the regional image may be an image corresponding to the region of interest ROI, and the control terminal may identify the region of interest of the vehicle image according to the vehicle area to be tested indicated by the detection item, and determine the region of interest of the vehicle area to be tested from the vehicle image. Exemplarily, as shown in FIG3 , the vehicle image is larger than the regional image, and the image corresponding to the vehicle area to be tested may be smaller than or equal to the regional image, so as to ensure that key image features are not missing and improve the detection accuracy. Among them, the regional image may be an image corresponding to the ROI area.

[0084] S204, performing image detection processing on the area image according to the detection content of the vehicle to be tested area in the detection item, and determining the project detection result of the vehicle to be tested corresponding to the detection item.

[0085] The test content refers to the specific content of the vehicle area to be tested under the test item. Taking the rear tire test item as an example, the specific content may include the rear tire specifications, rear tire brand, rear tire taper yellow mark, rear tire weight red mark, etc. Taking the rear wheel hub test item as an example, the specific content may include the rear wheel hub specifications, rear wheel hub color, rear wheel hub shape, rear wheel hub texture, etc.

[0086] The project detection result may represent the result of image detection for the regional image. The project detection result includes the content detection result of each detection content in the detection project. Further, the control device may combine multiple project detection results to obtain the detection result of the target scene.

[0087] Specifically, the control device may perform image detection processing on the area image according to the detection content of the vehicle to be tested area in the detection project, and determine the project detection result of the vehicle to be tested corresponding to the detection project.

[0088] In one embodiment, the control device can directly perform image detection processing on the regional image according to the detection method matching the detection content, and determine the project detection result corresponding to the detection item of the vehicle to be tested. Exemplarily, when the detection content is specification, the detection method can be to perform image detection processing on the regional image using an appearance detection model. When the detection content is color, the detection method can be to perform image detection processing on the regional image using a color detection model.

[0089] In one embodiment, the control device can determine the target image carrying the detection content from the area image according to the detection content of the vehicle to be tested area in the detection project. The control device can perform image detection processing on the target image to determine the project detection result of the vehicle to be tested corresponding to the detection project.

[0090] S206, combining the test results of each test item to determine the test result of the vehicle to be tested corresponding to the target scene.

[0091] Combination can mean splicing and sorting multiple project test results. For example, the project test results A, B and C are spliced ​​and sorted to obtain the test results ACB of the vehicle to be tested corresponding to the target scene. Since the test results often contain a large number of project test results, in order to improve the display effect of the more important project test results, multiple combination methods can be configured. There are many ways to combine. For example, combine the test results of each project by weight, combine the project tests of some test items, etc.

[0092] Specifically, the control device can determine the combined weights of the test results of each item, sort the test results of each test item according to the combined weights, and determine the test results of the vehicle to be tested corresponding to the target scene. Among the test results of the target scene, the test results of items with higher combined weights are sorted higher.

[0093] In one embodiment, the control device may further determine a project screening condition configured for each project test result, and screen a specific project that satisfies the project screening condition in each test project. The control device may aggregate the project test results of each specific project to determine the test result of the vehicle to be tested corresponding to the target scene. The combined weight of the project test results of the specific project may be greater than the preset weight.

[0094] In this embodiment, for each detection item, according to the vehicle area to be tested indicated by the detection item, the vehicle image is segmented, and the regional image of the vehicle area to be tested is determined from the vehicle image, which reduces the background feature amount and eliminates the interference of irrelevant features, which helps to improve the accuracy of subsequent detection results. According to the detection content of the vehicle area to be tested in the detection item, the regional image is image detected and processed to determine the project detection result of the vehicle to be tested corresponding to the detection item, and the detection result of the vehicle to be tested corresponding to the target scene is determined by combining the project detection results of each detection item. The detection result reflects the importance of each project detection result, which can improve the display effect of the detection results of more important projects.

[0095] In one embodiment, when the control device performs image detection processing on the area image according to the detection content of the vehicle area to be tested in the detection project to determine the project detection result of the vehicle to be tested corresponding to the detection project, the control device is configured to: determine the target image carrying the detection content from the area image according to the detection content of the vehicle area to be tested in the detection project, perform image detection processing on the target image, and determine the project detection result of the vehicle to be tested corresponding to the detection project.

[0096] The target image represents a portion of the image in the region image that carries the detection content. For example, the region image may be a rear image of a vehicle, and the target image may represent a portion of the image in the rear image that carries the taillights of the vehicle.

[0097] The image detection process may be any one of appearance detection process, color detection process, tire detection process and text detection process.

[0098] Specifically, the control device may detect the detection content for the vehicle to be tested area in the project, and determine the target image carrying the detection content from the area image.

[0099] In one embodiment, the control device may obtain the detection content for the vehicle to be tested area in the detection project. The control device may train the target image recognition model using the area image corresponding to the detection content as a positive sample and the image corresponding to other content except the detection content as a negative sample. The control device may determine the target image carrying the detection content from the area image according to the target image recognition model.

[0100] Specifically, the item detection result may include a color detection result, and the control device may perform color detection processing on the target image to determine the color detection result corresponding to the detection item to be tested. The item detection result may also include a text detection result, and the control device may perform text detection processing on the target image to determine the text detection result corresponding to the detection item to be tested.

[0101] In this embodiment, according to the detection content of the vehicle area to be tested in the detection project, the target image carrying the detection content is determined from the area image, and image detection processing is performed on the target image to determine the project detection results of the vehicle to be tested corresponding to the detection project. The project detection results include color detection results and text detection results. The project detection results are more comprehensive, which is conducive to improving the accuracy of subsequent detection results.

[0102] In one embodiment, the item detection results include text detection results. When the control device performs image detection processing on the target image and determines the item detection results of the vehicle to be tested corresponding to the detection item, it is configured to: perform text recognition on the target image, determine a text image area containing text content from the target image, perform correction processing on the text image in the text image area to obtain a corrected image, identify characters contained in the corrected image, and obtain the text detection results of the vehicle to be tested corresponding to the detection item.

[0103] The text image region may be a region containing text content. The text image may be an image corresponding to the text image region in the target image. The corrected image may be a corrected text image.

[0104] A character can be a glyph-like unit or a symbol, including letters, numbers, operation symbols, punctuation marks and other symbols. A character can also be a specific Chinese character or text in other languages.

[0105] The text detection result may include the recognized target character, the prediction score of the target character, and the corresponding sequence number of the target character.

[0106] There are many ways to recognize text in a target image. Specifically, the control device can perform semantic segmentation on the target image to determine a text image region containing text content from the target image. The control device can also perform image segmentation on the target image to determine a text image region containing text content from the target image.

[0107] In one embodiment, the control device may also pre-train a key point recognition model. The control device performs text recognition on the target image according to the key point recognition model to obtain a plurality of key points corresponding to the text content. The control device may determine a text image region containing text content from the target image according to each key point. The text image region may be formed by connecting a plurality of key points in sequence.

[0108] Correcting the text image may be to adjust the angle corresponding to the text content so that the text content maintains a horizontal angle or other preset angles. Specifically, the control device may adjust the arrangement direction of characters in the text content so that the arrangement direction maintains a horizontal angle or other preset angles, and determine the current text image as a corrected image.

[0109] In one of the embodiments, the key points can also be corrected. After obtaining multiple key points, the control device can perform homography transformation to horizontal imaging based on the four corner points detected by the key points to correct the viewing angle and determine the text image at this time as the corrected image.

[0110] There are many ways to recognize characters contained in a character correction image, for example, a text recognition model based on deep learning, video text recognition technology VTR, optical character recognition OCR, etc.

[0111] Recognizing characters included in the character correction image may be optical character recognition. Specifically, the control device may select the ppocr-v3 algorithm to recognize characters included in the character correction image and obtain text detection results corresponding to the detection items of the vehicle to be tested.

[0112] Exemplary, a visualization diagram of the text detection results is shown in FIG4. The target image includes a text image area and other areas, the text image area includes uncorrected text content, and the corrected image includes corrected text content. The control device can identify the characters contained in the corrected image, and obtain the target characters corresponding to the detection items of the vehicle to be tested, the predicted scores of the target characters, and the corresponding serial numbers of the target characters. Among them, the target characters include "manufacturer: A, model: A, energy type: B, cruising range: C", etc. The predicted score of the target character can indicate the accuracy of the prediction of the target character. The serial number corresponding to the target character can be the ranking of the target character in the prediction result.

[0113] In this embodiment, text recognition is performed on the target image, a text image area containing text content is determined from the target image, the text image in the text image area is corrected to obtain a corrected image, the characters contained in the corrected image are identified, and the text detection results corresponding to the detection items of the vehicle to be tested are obtained. Correcting the text content is conducive to obtaining more accurate text detection results.

[0114] In one embodiment, the project detection results also include color detection results. When the control device performs image detection processing on the target image and determines the project detection results of the vehicle to be tested corresponding to the detection item, it is also configured to: obtain reference color information configured for the detection content, perform color recognition on the target image, determine the target color information of the target image, and determine the color detection results of the vehicle to be tested corresponding to the detection item based on the information difference between the target color information and the reference color information.

[0115] The reference color information may be pre-configured HSV information. HSV information includes H (Hue): representing hue, indicating the type of color; S (Saturation): representing saturation, that is, the color ranges from dark to light. The value range is 0~255, indicating the purity or intensity of the color; V (Value): representing brightness, indicating the brightness of the color. The value range is 0~255. Brightness represents the brightness of the color. The reference color information may also be other color information such as RGB value or grayscale value.

[0116] For example, the reference color information configured for the detection content may be the reference color information of the rear tire, that is, the HSV information of the rear tire. The HSV information may reflect the distribution of the channel values ​​of the three channels H, S and V, such as the distribution of the hue of the rear tire image.

[0117] The target color information may be HSV information of the target image in the HSV color space, or may be other color information such as RGB value or grayscale value.

[0118] The information difference between the target color information and the reference color information may include a first channel information difference between the H channel information in the target color information and the H channel information of the reference color information, a second channel information difference between the S channel information in the target color information and the S channel information of the reference color information, and a third channel information difference between the V channel information in the target color information and the V channel information of the reference color information.

[0119] Specifically, the control device can determine the total information difference between the target color information and the reference color information based on the first channel information difference, the second channel information difference, and the third channel information difference. The control device can determine the color detection result of the vehicle to be tested corresponding to the detection item according to the total information difference and the information difference threshold. The color detection result includes color detection passing OK and color detection failing NG.

[0120] In one embodiment, the control device may obtain the channel weight of each channel information difference. The control device may perform weighted processing on the first channel information difference, the second channel information difference, and the third channel information difference according to each channel weight to obtain the total information difference between the target color information and the reference color information.

[0121] In one embodiment, the control device may also use a histogram comparison method (such as correlation coefficient, chi-square distance, histogram intersection, etc.) to quantify the difference between two similar channels. For example, the control device may quantify the difference in information of the first channel, quantify the difference in information of the second channel, and quantify the difference in information of the third channel.

[0122] In one embodiment, the control device may also compare the values ​​of two channels of the same type pixel by pixel and calculate the sum or average of the differences. The control device may determine the color detection result of the vehicle to be tested corresponding to the detection item based on the calculated sum or average of the differences and the information difference threshold.

[0123] In this embodiment, reference color information configured for the detection content is obtained, color recognition is performed on the target image, target color information of the target image is determined, and based on the information difference between the target color information and the reference color information, the color detection result of the vehicle to be tested corresponding to the detection item is determined, and color matching is performed on the color information to obtain accurate color detection results.

[0124] In one embodiment, the project detection results also include appearance detection results. When the control device performs image detection processing on the target image to determine the project detection results of the vehicle to be tested corresponding to the detection item, it is also configured to: obtain reference appearance information configured for the detection content, perform appearance recognition on the target image, obtain the target appearance information of the image, and determine the appearance detection results of the vehicle to be tested corresponding to the detection item based on the information difference between the target appearance information and the reference appearance information.

[0125] The reference appearance information may be pre-configured vehicle styling information. The vehicle styling information may be styling information of various components. For example, styling information of side skirt guards, styling information of side fenders, styling information of laser radar guards, and styling information of corner windows. More specifically, the styling information of side skirt guards includes a preset number of side skirt guards on the vehicle, shape information of the side skirt guards, and the like.

[0126] The target appearance information may represent the recognition result obtained by performing appearance recognition on the target image, and the target appearance information reflects the actual shape information of the vehicle.

[0127] The appearance inspection result indicates the inspection result for each component. For example, if the reference appearance information indicates that the number of laser radar guards is 4, and the target image is subjected to appearance recognition, and the target appearance information obtained indicates that the number of laser radar guards is 4, then the appearance inspection result of the appearance inspection item of the laser radar guard is OK, i.e., the inspection has passed. If the target appearance information indicates that the number of guards is not 4, then the appearance inspection result of the appearance inspection item of the laser radar guard is determined to be NG, i.e., the inspection has failed.

[0128] The information difference between the target appearance information and the reference appearance information may indicate a difference in the number of components or a difference in the shape of the components.

[0129] Specifically, there are multiple ways to perform appearance recognition on a target image. The control device can perform appearance recognition on the target image based on an appearance recognition model to obtain target appearance information of the image. The appearance recognition model can be a global appearance recognition model or a local appearance recognition model.

[0130] In one embodiment, the control device may perform appearance recognition on the target image according to a global appearance recognition model to obtain global appearance information of the image. The control device may perform appearance recognition on the target image according to a local appearance recognition model to obtain local appearance information of the image. The control device may fuse the global appearance information and the local appearance information to obtain the target appearance information of the image. The appearance recognition model may be a global appearance recognition model or a local appearance recognition model.

[0131] In this embodiment, reference appearance information configured for the detection content is obtained, appearance recognition is performed on the target image, and the target appearance information of the image is obtained. Based on the information difference between the target appearance information and the reference appearance information, the appearance detection result of the vehicle to be tested corresponding to the detection item is determined to obtain an accurate appearance detection result.

[0132] In one embodiment, the project detection results also include tire detection results. When the control device performs image detection processing on the target image and determines the project detection results of the vehicle to be tested corresponding to the detection item, it is also configured to: obtain a tire detection model configured for the detection content, perform tire recognition on the target image according to the tire detection model, and obtain the tire detection results of the vehicle to be tested corresponding to the detection item.

[0133] The tire detection model can be a target detection algorithm based on deep learning, such as YOLOv8 and YOLOv3. These models can automatically analyze tire information in the image. The tire information includes tire type, tire appearance, tire color, tire text and other information.

[0134] The tire detection model may also include tire-related structural detection models, such as wheel hub detection model, decorative cover detection model, decorative cap detection model, brake caliper detection module, ventilated disc detection model, tire detection model, etc.

[0135] The tire test results can be the tire specifications, tire brands, tire taper yellow mark, light and heavy red mark test results. It can also be the wheel hub specifications, color, shape, and grain test results. It can also be the LOGO category and color (shape) test results of the decorative cover. It can also be the LOGO category and color test results of the decorative cover. It can also be the color test of the rear brake caliper. The test results of the presence or absence of holes in the ventilation (brake) disc. The text test results of the tire (Emark number test results).

[0136] Specifically, the control device may perform tire recognition on the target image according to the tire detection model to obtain a tire detection result corresponding to the detection item of the vehicle to be tested.

[0137] In one embodiment, the control device can also obtain the basic tire detection results, the hub detection results, the decorative cover detection results, the decorative cover detection results, the brake caliper detection module, the ventilated disc detection results, and the tire detection results according to the tire detection model, the wheel hub detection model, the decorative cover detection model, the decorative cover detection model, the brake caliper detection module, the ventilated disc detection model, and the tire detection model. The control device can integrate the basic tire detection results, the wheel hub detection results, the decorative cover detection results, the decorative cover detection results, the brake caliper detection module, the ventilated disc detection results, and the tire detection results to obtain the tire detection results corresponding to the detection items of the vehicle to be tested.

[0138] In one of the embodiments, the control device judges the test results of the basic tire test results, the wheel hub test results, the decorative cover test results, the decorative cap test results, the brake caliper test module, the ventilated disc test results, and the tire test results respectively. If each test result is OK, it is determined that the tire test result of the vehicle to be tested corresponding to the test item is OK. If there is a test result of NG, it is determined that the tire test result of the vehicle to be tested corresponding to the test item is NG.

[0139] In this embodiment, a tire detection model configured for the detection content is obtained, tire recognition is performed on the target image according to the tire detection model, and the tire detection result corresponding to the detection item of the vehicle to be tested is obtained to obtain the overall tire detection result.

[0140] As shown in FIG5 , a vehicle detection method is also provided, including:

[0141] S502 , in response to a detection scene selection event, determining a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes.

[0142] S504: Determine the detection items in the target scene based on the type of the vehicle to be detected.

[0143] S506: Analyze the vehicle image according to the detection items to determine the detection result of the vehicle to be detected corresponding to the target scene.

[0144] In this embodiment, in response to a detection scene selection event, a target scene configured for the vehicle to be tested is determined from a plurality of candidate detection scenes, and a personalized target scene configured for the vehicle to be tested is determined, providing a basis for the subsequent determination of the detection items. Based on the model of the vehicle to be tested, the detection items under the target scene are determined, and the detection items reflect the personalized detection requirements for the vehicle to be tested. With the above system, because the determination of the detection items takes into account the model of the vehicle to be tested and the detection items reflect the personalized detection requirements for the vehicle to be tested, the vehicle image is analyzed according to the detection items, and the detection result corresponding to the target scene of the vehicle to be tested is obtained with higher accuracy.

[0145] In one embodiment, S504 includes obtaining multiple candidate items configured for the model of the vehicle to be tested; determining at least two selected items matching the target scene from each candidate item; determining the detection item under the target scene in response to a detection item determination event; and the detection item includes at least a portion of each selected item.

[0146] In one embodiment, S506 includes, for each detection item, performing image segmentation on the vehicle image according to the vehicle area to be tested indicated by the detection item, and determining the regional image of the vehicle area to be tested from the vehicle image; performing image detection processing on the regional image according to the detection content of the vehicle area to be tested in the detection item, and determining the project detection result of the vehicle to be tested corresponding to the detection item; and determining the detection result of the vehicle to be tested corresponding to the target scene by combining the project detection results of each detection item.

[0147] In one embodiment, image detection processing is performed on the area image according to the detection content of the vehicle area to be tested in the detection project to determine the project detection result of the vehicle to be tested corresponding to the detection project, including: determining a target image carrying the detection content from the area image according to the detection content of the vehicle area to be tested in the detection project; performing image detection processing on the target image to determine the project detection result of the vehicle to be tested corresponding to the detection project.

[0148] In one embodiment, the project detection results include text detection results; performing image detection processing on the target image to determine the project detection results of the vehicle to be tested corresponding to the detection item, including: performing text recognition on the target image to determine a text image area containing text content from the target image; performing correction processing on the text image in the text image area to obtain a corrected image; identifying characters contained in the corrected image to obtain the text detection results of the vehicle to be tested corresponding to the detection item.

[0149] In one embodiment, the project detection result also includes a color detection result; performing image detection processing on the target image to determine the project detection result of the vehicle to be tested corresponding to the detection item, including: obtaining reference color information configured for the detection content; performing color recognition on the target image to determine the target color information of the target image; based on the information difference between the target color information and the reference color information, determining the color detection result of the vehicle to be tested corresponding to the detection item.

[0150] In one embodiment, as shown in FIG6 , a full-body anti-fool-proof visual inspection method is also provided, comprising:

[0151] S602, in response to a detection scene selection event, determining a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes.

[0152] S604, obtaining multiple candidate items configured for the vehicle model to which the vehicle to be tested belongs.

[0153] S606: Determine at least two selected items matching the target scenario from among the candidate items.

[0154] S608, in response to the detection item determination event, determining the detection items under the target scene.

[0155] The detection items include at least a portion of each selected item.

[0156] Among them, as shown in the schematic diagram of the full-body anti-foolproofing scene item in Figure 7, the target scene can be the full-body anti-foolproofing scene item. Specifically, the full-body anti-foolproofing scene item includes appearance detection scene, color detection scene, tire detection scene, and text detection scene.

[0157] The detection items can be configured measurement point items included in different detection scenarios.

[0158] Exemplary, schematic diagram of the appearance and color detection scene measurement point items, the appearance and color detection scene measurement point items include: 1) body, roof, hood: color detection; 2) front bumper upper trim cover: color, car logo detection; 3) front / rear bumper: trim color, grille (shape), radar presence, radar color, number of radars, license plate frame size detection; 4) tail label: color, tail label detection; 5) spoiler: color, shape detection; 6) trunk lid: color, camera presence detection; 7) rear surround camera: presence detection; 8) left / right rear fog lamp: presence detection; 9) left / right exterior rearview mirror: shell color, base color, shape (shape), high-definition 360 camera presence detection; 10) left front / right front / left rear / right rear door handle: color, PEPS presence, decoration (shape), handle presence, PE unlocking logo detection; 11) charging port cover: style detection; 12) left / right side skirt guard: color , shape detection; 13) Left / right fender: color, shape detection; 14) LiDAR guard plate: presence detection; 15) Left / right rear corner window: presence or absence of bright strips, whether it is privacy glass detection; 16) Left / right rear door glass: whether it is privacy glass detection; 17) Left front / right front / left rear / right rear water barrier: material (chrome / high gloss / matte) detection 18) Left / right side decorative bright strips: presence or absence detection; 19) Left front / right front / left rear / right rear side 1. Door trim: color and shape detection; 20) Left front / right front / left rear / right rear wheel arch decoration: color and shape detection; 21) Steering wheel: color and shape detection; 22) Airbag: color detection; 23) Left / right front door guard: color detection; 24) Left front / right front / left rear / right rear glass guide groove trim: color detection; 25) A-pillar paint: color detection; 26) D-pillar trim / trim: color detection; 27) Sunroof: style detection.

[0159] Schematic diagram of the tire detection scenario measurement point items, which include: 1) Left front / right front / left rear / right rear tire: specification, brand, yellow taper mark, light and heavy red mark detection; 2) Left front / right front / left rear / right rear wheel hub: specification, color, shape, and texture detection; 3) Left front / right front / left rear / right rear decorative cover: LOGO category, color (shape) detection; 4) Left front / right front / left rear / right rear decorative cover: LOGO category and color detection; 5) Left front / right front / left rear / right rear brake caliper: color detection; 6) Left front / right front ventilated (brake) disc: detection of perforation; 7) Left front / right front / left rear / right rear tire Emark number: text detection.

[0160] The measurement points of text detection scenarios include: energy consumption label, fuel consumption label, nameplate (handheld), Emark number, VIN barcode, air conditioning refrigerant filling label, and sticker measurement points.

[0161] Among them, the energy consumption label includes sub-items: 1) Pure electric and hybrid models include: vehicle brand, manufacturer, vehicle model, energy type, vehicle curb weight, drive motor peak power, maximum design gross mass, operating power consumption, driving range, registration number, and activation date measurement point sub-items; 2) Hybrid models include: engine model, transmission type, displacement, maximum net power, fuel consumption, electric energy consumption, electric energy equivalent fuel consumption, and minimum state of charge fuel consumption measurement point sub-items; 3) Pure electric models include: energy consumption label measurement point sub-items.

[0162] Among them, the fuel consumption label includes the following sub-items: 1) Fuel vehicle models include: vehicle brand, manufacturer, vehicle model, engine model, transmission type, energy type, vehicle curb weight, displacement, maximum design gross mass, maximum net power, drive type, suburban operating conditions, urban operating conditions, comprehensive operating conditions, registration number, start date measurement point sub-items.

[0163] Among them, the nameplate (handheld) includes the following sub-items: 1) All domestic vehicle models include: manufacturer, vehicle model, maximum allowable total mass, number of passengers, manufacturing year and month measurement point sub-items; 2) Domestic new energy vehicle models include: drive motor model, drive motor peak power, power battery system rated power, power battery system rated capacity measurement point sub-items; 3) Domestic fuel vehicle models include: engine model, engine displacement, engine maximum net power measurement point sub-items; 4) Export and domestic models include: VIN number, brand measurement point sub-items; 5) Export models include: manufacturer, WVTA certificate number, maximum allowable total mass, total weight of the vehicle with trailer when towing, maximum technical weight allowed by the front axle, maximum technical weight allowed by the rear axle, manufacturing and design, model code, compliance with existing GSO standards, vehicle category, MPV measurement point sub-items.

[0164] Among them, the Emark number includes sub-items: left / right exterior mirror side turn signal, left / right front corner window glass, left / right front supplementary light (front position light / front turn signal / daytime running light), sunroof glass, front windshield - normal, rear windshield - tailgate glass upper / lower, right rear floating window, left front / right front / left rear / right rear door glass measurement point sub-items.

[0165] Among them, the sub-items included in the air conditioning refrigerant filling label are: refrigerant filling quantity and refrigerant category; the sub-items included in the VIN barcode are: front gear VIN barcode, cabin VIN barcode, and mark label; the sub-items included in the label are: motor label and engine label.

[0166] S610, for each detection item, according to the vehicle to-be-detected area indicated by the detection item, the vehicle image is segmented, and a regional image of the vehicle to-be-detected area is determined from the vehicle image.

[0167] Exemplarily, after obtaining the regional image, detection can be performed according to different scene flow algorithms to obtain detection results. The scene flow algorithms include: appearance detection scene flow algorithm, color detection scene flow algorithm, tire detection scene flow algorithm and text detection scene flow algorithm.

[0168] Among them, the first scene flow algorithm is the appearance detection scene flow algorithm. The appearance scene detection requirement needs to detect whether a certain part of the vehicle exists or whether the shape is correct. The following algorithm solutions can be provided: Solution 1: Input image → target detection → output result; Solution 2: Input image → relative ROI → target detection → output result. Among them, the target detection algorithm can choose yolox, or other open source target detection algorithms. Among them, the relative ROI area of ​​interest is just to crop a piece of the original image area of ​​interest and send it to the target detection algorithm for detection. For small target appearance detection, the solution 2 process can be selected. The premise is that the scene requires more accurate vehicle positioning.

[0169] Among them, the target detection training configuration is as follows (different configurations need to be changed according to different scenarios):

[0170] Number of categories (num_classes): 7; category name (classes_name): omitted; batch size (batch_size): 16; learning rate (learning_rate): 0.001; iteration period (epochs): 100; save model interval (save_period): 20; image size (img_size): 640; model size (model_size): m.

[0171] The second scene flow algorithm is the color detection scene flow algorithm.

[0172] Color scene detection requires detecting whether the color of a certain part of the vehicle is correct. The following algorithm solutions can be provided: Solution 1: input image → template matching → correction ROI → color matching → output result; Solution 2: input image → target detection → correction ROI → color matching → output result.

[0173] Among them, the template matching algorithm is based on angle / scale template matching, which requires pre-production of templates and then template training.

[0174] Among them, the template matching training configuration is as follows (different configurations need to be changed according to different scenarios): number of features (NumFeature): 1024; padding size (Padding): 768; category ID (ClassID): "angle"; template feature file save path (TemplatePath): ".\\angle_templ.yaml"; template information file save path (InfoPath): ".\\angle_info.yaml"; angle range (AngleRange): [-20, 20]; angle step (AngleStep): 0.25; scale range (ScaleRange): [1.0]; scale step (ScaleStep): 0.1.

[0175] Among them, the template matching test configuration is as follows (different configurations need to be changed according to different scenarios): number of features (NumFeature): 1024; padding size (Padding): 768; category ID (ClassID): "angle"; template feature file loading path (TemplatePath): ".\\angle_templ.yaml"; template information file loading path (InfoPath): ".\\angle_info.yaml"; sub-pixel edge extraction Gaussian threshold (Edges_Alpha): 1.0; sub-pixel edge extraction low threshold (Edges_ThLow): 3; sub-pixel edge extraction high threshold (Edges_ThHigh): 5; sub-pixel edge extraction mode (Edges_Mode): 1; matching confidence threshold (ConfThresh): 60; only take the top N matches (TopN): 1; whether to enable template positioning (IsLocate): true; template image path (TemplateImagePath): ".\\template.jpg".

[0176] Among them, correcting ROI: the current template and the positioning template after template matching calculate the homography transformation matrix, and correct the relative ROI.

[0177] Among them, the color matching configuration is as follows (different configurations need to be changed according to different scenarios): Feature Type (FeatureType): "mean", supports maximum feature (max), mean feature (mean), maximum feature under HSV channel (hsv_max), mean feature under HSV channel (hsv_mean); Loss Type (LossType): "l1", supports L1 loss (l1), L2 loss (l2); Similarity confidence threshold (ConfThresh): 100; Only take the top N matches (TopN): 1; Take the top N multi-templates (MultiTopN): 2;

[0178] Among them, the target detection training configuration is as follows (different configurations need to change according to different scenarios): number of categories (num_classes): 2; category name (classes_name): ['door', 'camera']; batch size (batch_size): 16; learning rate (learning_rate): 0.001; iteration cycle (epochs): 100; save model interval (save_period): 20; image size (img_size): 640; model size (model_size): m.

[0179] The third scene flow algorithm is the tire detection scene flow algorithm.

[0180] Tire scene detection requires the detection of information on the four tire-related measurement points of the vehicle. The following algorithm solutions can be provided: a) Tire detection: input image → target detection tire / wheel → cut image → rotate cut image → target detection tire-related measurement points → output results; b) Wheel detection: input image → target detection tire / wheel → cut → target detection wheel-related measurement points → output results.

[0181] Among them, the target detection algorithm can choose yolov8, or other open source target detection algorithms. The first target detection function is to locate tires and wheels. The training configuration is as follows (different configurations need to be changed according to different scenarios): number of categories (num_classes): 2; category name (classes_name): ['1tyre', '2hub']; batch size (batch_size): 8; learning rate (learning_rate): 0.01; iteration cycle (epochs): 500; save model interval (save_period): 100; image size (img_size): 640; model size (model_size): l.

[0182] Among them, the target detection tire-related measurement point training configuration is as follows (different configurations need to be changed according to different scenarios):

[0183] Number of categories (num_classes): 11; Category name (classes_name): omitted; Batch size (batch_size): 8; Learning rate (learning_rate): 0.01; Iteration cycle (epochs): 300; Save model interval (save_period): 50; Image size (img_size): 640; Model size (model_size): l; Among them, the training configuration of target detection wheel hub related measurement points is as follows (different configurations need to be changed according to different scenarios): Number of categories (num_classes): 9; Category name (classes_name): omitted; Batch size (batch_size): 8; Learning rate (learning_rate): 0.01; Iteration cycle (epochs): 300; Save model interval (save_period): 50; Image size (img_size): 640; Model size (model_size): l.

[0184] The fourth scene flow algorithm is the text detection scene flow algorithm.

[0185] Solution: Input image → key point detection → key point correction image → OCR recognition → output result.

[0186] In general text recognition scenarios, due to the limitation of the production line, the camera layout cannot shoot directly at the text detection item. In view of this situation, the above text detection scene flow is an algorithm solution applicable to all text perspectives.

[0187] The key point detection can choose yolov8-pose or other open source key point detection algorithms. The key point detection training configuration is as follows: bounding box class name (classes_name): ['BD']; key point class name (keypoint_class): ['LU', 'RU', 'RD', 'LD']; key point shape (kpt_shape): [4, 3]; flip index (flip_idx): [0, 1, 2, 3]; batch size (batch_size): 16; learning rate (img_size): 0.01; iteration cycle (epochs): l00; image size (img_size): 640; model size (model_size): n.

[0188] Among them, the key point correction image is transformed to a horizontal image based on the four corner points detected by the key points, so that the perspective is corrected to facilitate subsequent OCR (Optical Character Recognition) recognition.

[0189] For OCR recognition, you can choose ppocr-v3 or other open source OCR recognition algorithms. The OCR recognition training configuration is as follows:

[0190] Bounding box class name (classes_name): ['BD']; key point class name (keypoint_class): ['LU', 'RU', 'RD', 'LD']; key point shape (kpt_shape): [4, 3]; flip index (flip_idx): [0, 1,2, 3]; batch size (batch_size): 8; learning rate (img_size): 0.001; iteration period (epochs): l00; save model interval (save_period): 50; language (lang): ch # en means English, ch means Chinese.

[0191] Exemplarily, a text detection result includes text_num, texts, and scores, where text_num indicates the number of predicted texts, texts indicates the predicted texts, and scores indicates the scores of the predicted texts.

[0192] S612: According to the detection content of the vehicle to-be-tested area in the detection project, a target image carrying the detection content is determined from the area image.

[0193] S614, performing image detection processing on the target image to determine the item detection result corresponding to the detection item of the vehicle to be tested.

[0194] The project detection result also includes the appearance detection result. Specifically, the control device can obtain the reference appearance information configured for the detection content. The control device can perform appearance recognition on the target image to obtain the target appearance information of the image. The control device can determine the appearance detection result of the vehicle to be tested corresponding to the detection item based on the information difference between the target appearance information and the reference appearance information.

[0195] The item detection results also include tire detection results. Specifically, the control device can obtain a tire detection model configured for the detection content. The control device can perform tire recognition on the target image according to the tire detection model to obtain the tire detection result of the vehicle to be tested corresponding to the detection item.

[0196] S616, performing text recognition on the target image, and determining a text image region containing text content from the target image.

[0197] S618, performing correction processing on the text image in the text image area to obtain a corrected image.

[0198] S620, identifying characters contained in the corrected image, and obtaining text detection results corresponding to the detection items of the vehicle to be tested.

[0199] S622: Obtain reference color information configured for the detection content.

[0200] S624, performing color recognition on the target image to determine target color information of the target image.

[0201] S626, determining a color detection result of the vehicle to be tested corresponding to the detection item based on the information difference between the target color information and the reference color information.

[0202] Exemplarily, after obtaining various test results, various test results and corresponding visualizations of various test results can be displayed on the quality inspection page. The test results include OK, NG, and NT. Exemplary, a schematic diagram of a quality inspection page as shown in Figure 8. In this page, 47 test results were detected, of which 46 test results were OK and 1 test result was NG. The test result A of NG can be selected, and the details of the test result A can be displayed on the details page. The visualization of the test results can include key test information such as identification boxes and ROI areas for further verification by the verification personnel.

[0203] In this embodiment, in response to the detection scene selection event, a target scene configured for the vehicle to be tested is determined from multiple candidate detection scenes, and a personalized target scene can be configured for the vehicle to be tested, providing a basis for the subsequent determination of the detection items. Further, based on the model of the vehicle to be tested, the detection items under the target scene are determined, and the vehicle image is further analyzed according to the detection items to obtain the detection results. With the above method, since the determination of the detection items takes into account the model of the vehicle to be tested and the personalized detection requirements for the vehicle to be tested, the detection results obtained by analyzing the vehicle image according to the detection items are highly accurate. With the above method (1) the scene is no longer single, and all customer measurement points are fully covered through the characteristics of the whole vehicle, and customer needs are divided into four types of scene algorithms: appearance scene algorithm, color scene algorithm, tire scene algorithm, and text scene algorithm; (2) it is highly scalable and simple. According to the customer's new measurement point requirements, it can be planned into the required scene algorithm, and can be directly expanded to complete algorithm adaptation and output results. (3) It has high versatility, which is equivalent to the standardization of the full-body anti-fool algorithm, and can seamlessly connect the measurement point requirements of all models of all manufacturers. (4) It realizes the full-process automated visual inspection of the entire vehicle, including four major scene flows: appearance inspection, color inspection, tire inspection and text inspection, covering hundreds of measurement points; it is highly scalable and can add new measurement points based on existing scene algorithms; it is highly versatile and can seamlessly connect to the measurement point requirements of all models of all manufacturers.

[0204] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0205] Based on the same inventive concept, the embodiment of the present application also provides a vehicle detection device for implementing the vehicle detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle detection device embodiments provided below can refer to the limitations on the vehicle detection method above, and will not be repeated here.

[0206] In one embodiment, as shown in FIG. 9 , a vehicle detection device 900 is provided, including: a scene determination module 902 , a detection item determination module 904 , and an image analysis module 906 , wherein:

[0207] The scene determination module 902 is used to determine a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes in response to a detection scene selection event.

[0208] The detection item determination module 904 is used to determine the detection items in the target scene based on the type of vehicle to be tested.

[0209] The image analysis module 906 is used to analyze the vehicle image according to the detection items and determine the detection result of the vehicle to be detected corresponding to the target scene.

[0210] In one embodiment, the detection item determination module 904 is also used to obtain multiple candidate items configured for the model of the vehicle to be tested; determine at least two selected items that match the target scene from each candidate item; determine the detection items under the target scene in response to a detection item determination event; the detection items include at least a part of each selected item.

[0211] In one embodiment, the image analysis module 906 is also used to perform image segmentation on the vehicle image for each detection item according to the vehicle area to be tested indicated by the detection item, and determine the regional image of the vehicle area to be tested from the vehicle image; perform image detection processing on the regional image according to the detection content of the vehicle area to be tested in the detection item, and determine the project detection result of the vehicle to be tested corresponding to the detection item; and determine the detection result of the vehicle to be tested corresponding to the target scene by combining the project detection results of each detection item.

[0212] In one embodiment, the image analysis module 906 is also used to determine a target image carrying the detection content from the area image according to the detection content of the vehicle to be tested area in the detection project; perform image detection processing on the target image to determine the project detection result of the vehicle to be tested corresponding to the detection project.

[0213] In one embodiment, the project detection results include text detection results; the image analysis module 206 is also used to perform text recognition on the target image, determine the text image area containing text content from the target image; perform correction processing on the text image in the text image area to obtain a corrected image; identify the characters contained in the corrected image to obtain the text detection result of the vehicle to be tested corresponding to the detection item.

[0214] In one embodiment, the project detection results also include color detection results; the image analysis module 206 is also used to obtain reference color information configured for the detection content; perform color recognition on the target image to determine the target color information of the target image; based on the information difference between the target color information and the reference color information, determine the color detection result of the vehicle to be tested corresponding to the detection item.

[0215] Each module in the above vehicle detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0216] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store detection result data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a vehicle detection method is implemented.

[0217] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0218] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method steps are implemented when the processor executes the computer program.

[0219] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0220] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above method steps when executed by a processor.

[0221] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0222] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0223] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A vehicle detection system, characterized in that: The system comprises: Conveying equipment, used to transfer the vehicle to be tested to the vehicle testing station; An image acquisition device is used to acquire a vehicle image of the vehicle to be tested at the vehicle detection station; Control equipment for: In response to a detection scene selection event, determining a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes; Determining the detection items under the target scene based on the model of the vehicle to be tested; The vehicle image is analyzed according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

2. The system according to claim 1, characterized in that When determining the detection items in the target scene based on the type of vehicle to be tested, the control device is configured to: Acquire multiple candidate items configured for the vehicle model to which the vehicle to be tested belongs; Determining at least two selected items matching the target scenario from among the candidate items; In response to a detection item determination event, a detection item in the target scene is determined; the detection item includes at least a portion of each of the selected items.

3. The system according to claim 1, characterized in that The number of the detection items is multiple; When the control device analyzes the vehicle image according to the detection items and determines the detection result of the vehicle to be tested corresponding to the target scene, the control device is configured to: For each of the detection items, according to the vehicle to-be-detected area indicated by the detection item, the vehicle image is segmented, and a regional image of the vehicle to-be-detected area is determined from the vehicle image; According to the detection content of the vehicle to be tested area in the detection project, image detection processing is performed on the area image to determine the project detection result of the vehicle to be tested corresponding to the detection project; The detection result of the vehicle to be tested corresponding to the target scene is determined by combining the detection results of each of the detection items.

4. The system according to claim 3, characterized in that When the control device performs image detection processing on the area image according to the detection content of the vehicle to be tested area in the detection item and determines the item detection result of the vehicle to be tested corresponding to the detection item, the control device is configured to: According to the detection content of the vehicle to-be-tested area in the detection project, determining a target image carrying the detection content from the area image; Perform image detection processing on the target image to determine the item detection result of the vehicle to be tested corresponding to the detection item.

5. The system according to claim 4, characterized in that The project detection results include text detection results; When the control device performs image detection processing on the target image to determine the item detection result of the vehicle to be tested corresponding to the detection item, the control device is configured to: Performing text recognition on the target image to determine a text image region containing text content from the target image; Performing correction processing on the text image in the text image area to obtain a corrected image; The characters contained in the correction image are recognized to obtain a text detection result of the vehicle to be tested corresponding to the detection item.

6. The system according to claim 4, characterized in that The project detection results also include color detection results; When the control device performs image detection processing on the target image to determine the item detection result of the vehicle to be tested corresponding to the detection item, the control device is further configured to: Acquire reference color information configured for the detection content; Performing color recognition on the target image to determine target color information of the target image; Based on the information difference between the target color information and the reference color information, a color detection result of the vehicle to be tested corresponding to the detection item is determined.

7. The system according to claim 4, characterized in that The project test results also include appearance test results; When the control device performs image detection processing on the target image to determine the item detection result of the vehicle to be tested corresponding to the detection item, the control device is further configured to: Acquire reference appearance information configured for the detection content; Performing appearance recognition on the target image to obtain target appearance information of the image; Based on the information difference between the target appearance information and the reference appearance information, an appearance detection result of the vehicle to be tested corresponding to the detection item is determined.

8. The system according to claim 4, characterized in that The project test results also include tire test results; When the control device performs image detection processing on the target image to determine the item detection result of the vehicle to be tested corresponding to the detection item, the control device is further configured to: Acquire a tire detection model configured for the detection content; The target image is subjected to tire recognition according to the tire detection model to obtain a tire detection result of the vehicle to be tested corresponding to the detection item.

9. A vehicle detection method, characterized in that: Applied to the control device according to any one of claims 1 to 8, the method comprises: In response to a detection scene selection event, determining a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes; Determining the detection items under the target scene based on the model of the vehicle to be tested; The vehicle image is analyzed according to the detection items to determine the detection result of the vehicle to be tested corresponding to the target scene.

10. A vehicle detection device, characterized in that: The device comprises: A scene determination module, for determining a target scene configured for the vehicle to be tested from a plurality of candidate detection scenes in response to a detection scene selection event; A detection item determination module, used to determine the detection items in the target scene based on the type of vehicle to be tested; The image analysis module is used to analyze the vehicle image according to the detection items and determine the detection result of the vehicle to be detected corresponding to the target scene.

Images