A vehicle cleanliness detection method, device and computer readable storage medium

Abstract

The application discloses a vehicle cleanliness detection method, device and computer readable storage medium, the vehicle cleanliness detection method comprises: acquiring a to-be-detected vehicle image; segmenting the to-be-detected vehicle image to obtain a plurality of vehicle component images, the vehicle component image being an image in which a key component of a vehicle in the to-be-detected vehicle image is located; extracting features of the vehicle component images by using a feature extraction model in a cleanliness identification model to obtain global features and local features; fusing the global features and the local features by using a cleanliness prediction model in the cleanliness identification model to obtain fused features, and predicting the fused features to obtain a cleanliness score of each vehicle component image. In the foregoing manner, the application can improve the accuracy of vehicle cleanliness detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a vehicle cleanliness detection method and device and computer readable storage medium. BACKGROUND

[0002] At present, vehicle cleanliness recognition technology mainly relies on manual selection of relatively easy-to-judge parts in the vehicle as reference points, and then uses traditional image sharpness judgment methods to detect vehicle cleanliness. However, this method can only roughly judge the cleanliness according to one or two vehicle characteristics, and is easy to lose the detailed features of many vehicle parts, resulting in low accuracy of vehicle cleanliness detection. SUMMARY

[0003] The present application provides a vehicle cleanliness detection method, device and computer readable storage medium, which can improve the accuracy of vehicle cleanliness detection.

[0004] To solve the above technical problems, the technical scheme adopted by the present application is: providing a vehicle cleanliness detection method, the vehicle cleanliness detection method comprising: acquiring a vehicle image to be detected; segmenting the vehicle image to be detected to obtain a plurality of vehicle component images, the vehicle component image being an image in which a key component of the vehicle in the vehicle image to be detected is located; using a feature extraction model in a cleanliness recognition model to extract features of the vehicle component image to obtain global features and local features; using a cleanliness prediction model in the cleanliness recognition model to fuse the global features and the local features to obtain fused features, and predicting the fused features to obtain a cleanliness score of each vehicle component image.

[0005] To solve the above technical problems, another technical scheme adopted by the present application is: providing a vehicle cleanliness detection device, the vehicle cleanliness detection device comprising a memory and a processor connected to each other, wherein the memory is used to store a computer program, and the computer program is used to implement the vehicle cleanliness detection method in the above technical scheme when executed by the processor.

[0006] To solve the above technical problems, still another technical scheme adopted by the present application is: providing a computer readable storage medium, which is used to store a computer program, and the computer program is used to implement the vehicle cleanliness detection method in the above technical scheme when executed by the processor.

[0007] By the above scheme, the beneficial effects of the present application are: obtaining a cleanliness identification model, the cleanliness identification model comprising a feature extraction model and a cleanliness prediction model; segmenting a to-be-detected vehicle image to obtain a plurality of vehicle component images; then using the feature extraction model to extract features of the vehicle component images to obtain global features and local features; using the cleanliness prediction model to fuse the global features and the local features to obtain fused features, and predicting the fused features to obtain a cleanliness score of each vehicle component image; by segmenting key components of a vehicle in the to-be-detected vehicle image, and then respectively predicting the cleanliness of the images in which the plurality of vehicle components are located, the cleanliness of the entire vehicle is predicted, which can greatly improve the detection accuracy of vehicle cleanliness; meanwhile, the global features and the multi-scale local features of the vehicle component images are fused, and the overall information and the detailed information of the vehicle components are captured, so as to use the fused features obtained by fusing the global features and the local features to detect the cleanliness, improve the accuracy of the cleanliness detection of the vehicle component images, and realize high-precision identification of the vehicle cleanliness. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0009] Figure 1 is a flowchart of an embodiment of the vehicle cleanliness detection method provided by the present application;

[0010] Figure 2 is a flowchart of another embodiment of the vehicle cleanliness detection method provided by the present application;

[0011] Figure 3 is a structural diagram of the cleanliness identification model provided by the present application;

[0012] Figure 4 is a flowchart of an embodiment of the training method of the cleanliness identification model provided by the present application;

[0013] Figure 5 is a structural diagram of an embodiment of the vehicle cleanliness detection device provided by the present application;

[0014] Figure 6 is a structural diagram of an embodiment of the computer readable storage medium provided by the present application. DETAILED DESCRIPTION

[0015] The application will be described in further detail below with reference to the drawings and embodiments. It is particularly pointed out that the following embodiments are only for illustration of the application and do not limit the scope of the application. Similarly, the following embodiments are only part of the embodiments of the application, and all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the application.

[0016] In the present application, the term "embodiment" means that the specific features, structures or properties described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0017] It should be noted that the terms "first", "second", "third" in the present application are only for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0018] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the vehicle cleanliness detection method provided by the present application, which comprises:

[0019] Step 11: obtaining a vehicle image to be detected.

[0020] The vehicle image to be detected can include a picture of the vehicle to be detected, and the vehicle image to be detected can be obtained by a collection device, which can be a monitoring camera, without limitation.

[0021] Step 12: segmenting the vehicle image to be detected to obtain a plurality of vehicle component images.

[0022] The vehicle component image is an image in which a key component of a vehicle in the to-be-detected vehicle image is located. The key component of the vehicle can include a wheel, a front or rear door, a window, or the like. Specifically, the vehicle key component detection model can be used to detect the to-be-detected vehicle image to obtain key component information. Then, based on the key component information, the to-be-detected vehicle image is segmented into a plurality of vehicle component images. The vehicle key component information can include coordinate information of the key component of the vehicle in the to-be-detected vehicle image, so that the image in which the vehicle component is located in the to-be-detected vehicle image is segmented based on the coordinate information, and a plurality of vehicle component images are obtained.

[0023] Further, the vehicle key component detection model can include a backbone network and a detection network. The backbone network can be used to extract features of the to-be-detected vehicle image to obtain image feature information. Then, the detection network is used to detect the image feature information to obtain the key component information. The backbone network can include a residual network (for example, ResNet-50 residual network). The detection network can be a single-stage detector, for example, a single-stage multi-box detector (SSD), a YOLO (You Only Look Once), or a Retina Network (RetinaNet), or the like. Alternatively, the detection network can also be a common candidate region-based detector, for example, a Fast R-CNN (Region Convolutional Neural Networks), a Faster R-CNN, or a Feature Pyramid Network (FPN), or the like, without limitation.

[0024] Step 13: using a feature extraction model in the cleanliness recognition model to extract features of the vehicle component image to obtain global features and local features.

[0025] The cleanliness recognition model can include a feature extraction model and a cleanliness prediction model. The feature extraction model can be used to extract features of the vehicle component image to obtain global features and local features. The feature extraction model can be a conventional network structure in the field of deep learning technology, for example, a ResNet-50 residual network, without limitation. Specifically, the global features can be semantic features of the vehicle in the vehicle component image, for example, a color, a shape, or a texture of the vehicle, or the like. The local features can be detailed features of a local region in the vehicle component image, for example, features of a door handle region in a door image.

[0026] Further, the feature extraction model can include a plurality of convolutional layers connected in sequence, and the plurality of convolutional layers connected in sequence can be used to extract features of the vehicle component image to obtain global features; the feature maps output by part of the convolutional layers can be determined as local features; it can be understood that, due to different sizes of features obtained by each convolutional layer, multi-scale local features can be output by part of the convolutional layers to improve the accuracy of subsequent cleanliness detection.

[0027] In an embodiment, taking the ResNet-50 residual network as an example of the feature extraction model, the ResNet-50 residual network includes 50 convolutional layers connected in sequence, and the 50 convolutional layers connected in sequence can be used to extract features of the vehicle component image to obtain global features; accordingly, the features output by the conv2_10 layer (i.e., the tenth convolutional layer in the second convolutional block), the conv3_12 layer (i.e., the twelfth convolutional layer in the third convolutional block), and the conv4_18 layer (i.e., the eighteenth convolutional layer in the fourth convolutional block) in the ResNet-50 residual network can be determined as local features; it can be understood that, which convolutional layers in the ResNet-50 residual network are used to output local features can be selected according to actual conditions or experience, which is not limited herein.

[0028] Step 14: fusing the global features and the local features by using a cleanliness prediction model in the cleanliness recognition model to obtain fused features, and predicting the fused features to obtain a cleanliness score of each vehicle component image.

[0029] The global features and the local features are fused by using the cleanliness prediction model to obtain fused features, and the fused features are predicted to obtain a cleanliness score of each vehicle component image, so that the overall information and the detailed information of the vehicle component are captured simultaneously by the global features and the local features, the accuracy of describing the features of the vehicle component image is improved, and the fused features are used for cleanliness detection, which can greatly improve the accuracy of the cleanliness detection of the vehicle component image.

[0030] The embodiment can greatly improve the detection accuracy of the vehicle cleanliness by segmenting the key components of the vehicle in the vehicle image to be detected, and then performing cleanliness prediction on the images in which the plurality of vehicle components are located to complete the cleanliness prediction of the whole vehicle; meanwhile, the global features of the vehicle component image and the multi-scale local features are fused, and the overall information and the detailed information of the vehicle component are captured, so that the fused features obtained by fusing the global features and the local features are used for cleanliness detection to improve the accuracy of the cleanliness detection of the vehicle component image, and high-precision recognition of the vehicle cleanliness is realized.

[0031] Please refer to Figure 2 , Figure 2is a flowchart of another embodiment of the vehicle cleanliness detection method provided in the present application, and the method comprises the following steps:

[0032] Step 21: Obtain an original vehicle image, and detect the original vehicle image by using a vehicle type identification model to obtain the type of the vehicle in the original vehicle image.

[0033] Obtain an original vehicle image, and detect the original vehicle image by using a vehicle type identification model to obtain the type of the vehicle in the original vehicle image. Specifically, the original vehicle image can be an image containing a vehicle obtained by a collection device. Taking a monitoring camera as an example, video data in a monitoring scene collected by the monitoring camera can be processed, and an image of a vehicle region in the video data is cropped to obtain the original vehicle image.

[0034] The type of the vehicle (denoted as vehicle type) can include a truck, a bus, a public bus, or a family car. The family car can include a sedan, a sport utility vehicle (suv), or a multi-Purpose Vehicles (mpv), etc. The vehicle type identification model can be used to detect the original vehicle image to obtain the vehicle type of the vehicle in the original vehicle image. Specifically, the vehicle type identification model can include a data input layer, a feature extraction layer, and a classification layer. The feature extraction layer can include a sub-feature extraction module and an attention module. The number of attention modules can be one, two, or more than two. Each attention module can be arranged between two sub-feature extraction modules to enhance the features output by the sub-feature extraction modules. The number and position of the attention modules can be set according to actual application requirements, which are not limited herein. It can be understood that the sub-feature extraction module can be a convolution layer, and the classification layer can be a full connection layer. In other embodiments, the vehicle type identification model can also include a drop_out layer or a classification loss layer, etc. The specific network structure of the vehicle type identification model is not limited herein.

[0035] In a specific embodiment, in order to detect the original vehicle image by using the vehicle type identification model to obtain the type of the vehicle in the original vehicle image, the data input layer can be used to input the original vehicle image to the sub-feature extraction module. The sub-feature extraction module can be used to extract features of the original vehicle image to obtain original image features. The attention module can be used to enhance the original image features to obtain enhanced features. The classification layer can be used to classify the enhanced features to obtain the type of the vehicle.

[0036] The attention module can include a feature compression module, a feature excitation module, and a feature re-labeling module. The feature compression module can be used to perform global average pooling on original image features, compress the two-dimensional original image features into a real number, the real number has a receptive field of global features, and the dimension and the number of feature channels are consistent, which can represent the global distribution of the original image features in the feature channel; the feature excitation module includes a first fully connected layer and a second fully connected layer, the first fully connected layer can be used to reduce the number of feature channels and reduce the amount of calculation, and a rectified linear unit (ReLU) is used to process the real number to obtain a nonlinear feature, the second fully connected layer is used to perform dimension increasing processing on the nonlinear feature, and a Sigmoid activation function is used to calculate a normalized weight between 0 and 1 for each feature channel, the normalized weight is used to represent the importance of each feature channel; the feature re-labeling module uses the normalized weight output by the feature excitation module to weight the original image features to obtain enhanced features, thereby completing the re-labeling of the original image features in the feature channel dimension, and obtaining the most representative image features to improve the accuracy of the vehicle model recognition model.

[0037] Step 22: Determine whether the type of the vehicle is a preset vehicle model.

[0038] The vehicle type recognized by the vehicle model recognition model can be used to filter the original vehicle image to obtain a vehicle image to be detected of the vehicle under the preset vehicle model; the preset vehicle model can be customized according to actual application requirements, for example, if the preset vehicle model is a bus, it means that the embodiment is to detect the cleanliness of the bus, if it is applied to the cleanliness detection of a large bus, the preset vehicle model can be set to a large bus to filter the vehicle image to be detected containing the large bus, which is not limited herein.

[0039] Step 23: If the type of the vehicle is the preset vehicle model, determine that the original vehicle image is the vehicle image to be detected.

[0040] When the type of the vehicle is the preset vehicle model, it means that the vehicle model contained in the original vehicle image meets the current detection requirement, and then the original vehicle image is determined as the vehicle image to be detected; if the type of the vehicle is not the preset vehicle model, return to the step of obtaining the original vehicle image until the vehicle image to be detected is generated.

[0041] Step 24: Segment the vehicle image to be detected to obtain a plurality of vehicle component images.

[0042] The vehicle component image is an image of a key component of the vehicle in the vehicle image to be detected, wherein step 24 is the same as step 12 in the above embodiment, and will not be repeated here.

[0043] Step 25: feature extraction is performed on the vehicle component image by using the feature extraction model in the cleanliness recognition model to obtain global features and local features.

[0044] Step 25 is the same as step 13 in the above embodiment, and will not be repeated here.

[0045] In a specific embodiment, as shown in Figure 3 , Figure 3 is a structural diagram of the cleanliness recognition model, which includes a feature extraction model and a cleanliness prediction model. The feature extraction model can include a local feature extraction module, a plurality of sequentially connected convolutional layers, and a global average pooling layer. The plurality of sequentially connected convolutional layers and the global average pooling layer are used to extract features from the vehicle component image to obtain global features. The local feature extraction module extracts feature maps output by some of the convolutional layers to obtain local features.

[0046] Step 26: the global features and the local features are fused by using the cleanliness prediction model in the cleanliness recognition model to obtain fused features, and the fused features are predicted to obtain a cleanliness score of each vehicle component image.

[0047] The global features and the local features are fused by using the cleanliness prediction model to obtain fused features, and the fused features are predicted to obtain a cleanliness score of each vehicle component image. Specifically, the cleanliness prediction model can include a feature concatenation layer and a plurality of sequentially connected fully connected layers. The feature concatenation layer can be used to fuse the global features and the local features to obtain the fused features. The plurality of sequentially connected fully connected layers are used to predict the fused features to obtain the cleanliness score. For example, as shown in Figure 3 , the cleanliness prediction model can include four fully connected layers. The fused features output by the feature concatenation layer are taken as inputs of the four fully connected layers. The four fully connected layers output the cleanliness score of the vehicle component image through forward propagation.

[0048] Step 27: all cleanliness scores are weighted and summed to obtain an overall cleanliness score.

[0049] All cleanliness scores predicted by the vehicle component image are weighted and summed to obtain an overall cleanliness score. Specifically, a preset weight mapping table is obtained, and then the type of the key component is matched with the preset weight mapping table to obtain a weight value corresponding to the vehicle component image. Thus, based on the weight value corresponding to the vehicle component image, the plurality of cleanliness scores are weighted and summed to obtain a more accurate overall cleanliness score.

[0050] Further, the preset weight mapping table can be set according to actual conditions, which can include the corresponding relationship between the types of key components and the weight values, and the weight values are positively correlated with the importance of the key components of the vehicle; for example, because the wheels often come into contact with the ground, the wheels are often covered with more dust and have lower cleanliness, and therefore, compared with key components such as vehicle windows or vehicle doors, the weight value of the wheels can be set to be lower to reduce the influence of the wheels on the overall cleanliness score.

[0051] Step 28: determining whether the overall cleanliness score exceeds a preset threshold.

[0052] The cleanliness score is positively correlated with the cleanliness of the vehicle, that is, the higher the cleanliness score, the cleaner the vehicle, and whether the vehicle needs to be cleaned is determined by determining whether the overall cleanliness score exceeds a preset threshold.

[0053] Step 29: if the overall cleanliness score is less than or equal to the preset threshold, it is determined that the vehicle needs to be cleaned.

[0054] If the overall cleanliness score is less than or equal to the preset threshold, it is determined that the vehicle needs to be cleaned; if the overall cleanliness score exceeds the preset threshold, it is determined that the vehicle does not need to be cleaned; specifically, the preset threshold can be set according to actual conditions, which is not limited herein, for example, the cleanliness score has a value range of 0-100, and the preset threshold can be set to 80, that is, when the overall cleanliness score is greater than 80, it is determined that the vehicle does not need to be cleaned.

[0055] In a specific embodiment, as shown in FIG. 1, the step of training the cleanliness recognition model can include: Figure 4

[0056] Step 41: obtaining a vehicle cleanliness training set.

[0057] The vehicle cleanliness training set can include vehicle component sample images and label values corresponding to the vehicle component sample images, and the label values corresponding to the vehicle component sample images are artificially labeled cleanliness scores; wherein the size of all vehicle component sample images is a preset size, the preset size can include a preset length and a preset width, and the preset length and the preset width are equal, for example, the preset size can be 224*224; specifically, the step of obtaining the vehicle cleanliness training set can include: obtaining an original sample image, and determining whether the size of the original sample image is the preset size; if the size of the original sample image is the preset size, the original sample image is determined as a vehicle component sample image; if the size of the original sample image is not the preset size, a ratio of the length of the original sample image to the preset length is calculated to obtain a first ratio, and a ratio of the width of the original sample image to the preset width is calculated to obtain a second ratio; and the original sample image is processed based on the first ratio and the second ratio to generate a vehicle component sample image.

[0058] ​Further, when performing image size conversion, it is necessary to ensure that the length and width are scaled proportionally to prevent changes in image content. When the length and width of the original sample image are equal and the size of the original sample image is not the preset size, the original sample image can be directly proportionally adjusted. That is, when the first ratio is equal to the second ratio, the first ratio / second ratio is determined as the scaling ratio, and the original sample image is scaled according to the scaling ratio to obtain a vehicle component sample image.

[0059] When the length and width of the original sample image are not equal and the size of the original sample image is not the preset size, if the proportionally scaled vehicle component sample image with the preset length and the preset width is to be obtained, the image with the smaller size obtained after proportional scaling can be filled to obtain a vehicle component sample image with the preset proportion. Understandably, when the first ratio is less than the second ratio, the first ratio can be determined as the scaling ratio, the original sample image is scaled according to the scaling ratio to obtain a first scaled sample image, and the first scaled sample image is filled to obtain a vehicle component sample image. When the first ratio is greater than the second ratio, the second ratio is determined as the scaling ratio, the original sample image is scaled according to the scaling ratio to obtain a second scaled sample image, and the second scaled sample image is filled to obtain a vehicle component sample image.

[0060] In other embodiments, after obtaining the vehicle component sample image, the vehicle component sample image can be subjected to data processing operations such as random rotation, mirroring, or translation to expand the number of samples in the vehicle cleanliness training set, thereby preventing overfitting of the cleanliness recognition model.

[0061] Step 42: processing the vehicle component sample image using the cleanliness recognition model to obtain a cleanliness prediction score.

[0062] Step 42 is similar to steps 25-26 in the above embodiments and will not be described again here.

[0063] Step 43: calculating the cleanliness prediction score and the corresponding label value to obtain a training loss.

[0064] During the training of the cleanliness recognition model, the cleanliness prediction score and the corresponding label value can be calculated to obtain a training loss. Specifically, the training loss can be calculated using the following formula:

[0065]

[0066] In the above formula (1), L represents the training loss, N represents the number of samples in the vehicle cleanliness training set, pi represents the i-th vehicle component sample image, and V i represents the label value corresponding to the i-th vehicle component sample image. pia fusion feature representing the i-th vehicle component sample image, a cleanliness prediction score Q representing the i-th vehicle component sample image, i a label value corresponding to the i-th vehicle component sample image.

[0067] Step 44: determining whether the cleanliness identification model meets a training end condition based on the training loss.

[0068] The training end condition can include: loss convergence, i.e., a difference between a previous training loss and a current training loss value is less than a set value; determining whether the current training loss value is less than a preset loss, which is a preset loss threshold, and if the current training loss value is less than the preset loss, it is determined that the training end condition is met. Understandably, the training end condition can also include: the number of training reaches a set value (for example: training 10000 times); or the accuracy obtained when using the test set for testing reaches a set condition (such as: exceeding a preset accuracy), etc., which is not limited here.

[0069] Step 45: if the cleanliness identification model does not meet the training end condition, adjusting the model parameters of the cleanliness identification model.

[0070] When the cleanliness identification model does not meet the training end condition, the model parameters of the cleanliness identification model can be adjusted, and the step of processing the vehicle component sample image using the cleanliness identification model is returned until the cleanliness identification model meets the training end condition; when the cleanliness identification model meets the training end condition, the training can be ended, and the trained cleanliness identification model is obtained.

[0071] In a specific embodiment, the vehicle type identification model can also be trained using a vehicle type training set, and the key component detection model can be trained using a vehicle key component training set. The vehicle type training set can include vehicle sample images and corresponding vehicle type labels, and the vehicle key component training set can include vehicle component sample images and corresponding key component type labels. The acquisition process of the vehicle type training set and the vehicle key component training set and the training process of the vehicle type identification model and the key component detection model are similar to the above operations performed on the cleanliness identification model, and will not be described in detail here.

[0072] The embodiment adds an attention module in the vehicle type identification model, can better extract vehicle type features, and improve the accuracy of vehicle type identification; and can customize a preset vehicle type according to actual application requirements, so as to accurately screen the vehicle image to be detected by using the vehicle type identification model, to meet the cleanliness detection requirements of a specific vehicle type, and has universal applicability; in addition, by setting the vehicle key component detection model, the positions of the vehicle key components can be more accurately obtained, so that accurate multiple vehicle component images are obtained, to improve the accuracy of subsequent vehicle cleanliness identification, and the cleanliness identification model using multi-scale feature fusion is used to more accurately predict the cleanliness of each vehicle component, and the cleanliness scores of each key component are used to comprehensively obtain the vehicle cleanliness score, to further improve the accuracy of vehicle cleanliness identification of the preset vehicle type.

[0073] Referring to Figure 5 , Figure 5 is a structural schematic diagram of an embodiment of a vehicle cleanliness detection device provided by the present application. The vehicle cleanliness detection device 50 comprises a memory 51 and a processor 52 connected to each other. The memory 51 is configured to store a computer program. The computer program, when executed by the processor 52, is configured to implement the vehicle cleanliness detection method in the above embodiments.

[0074] Referring to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of a computer readable storage medium provided by the present application. The computer readable storage medium 60 is configured to store a computer program 61. The computer program 61, when executed by a processor, is configured to implement the vehicle cleanliness detection method in the above embodiments.

[0075] The computer readable storage medium 60 can be a server, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0076] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0077] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0078] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0079] If the technical scheme of the present application involves personal information, the product applying the technical scheme of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical scheme of the present application involves sensitive personal information, the product applying the technical scheme of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the person voluntarily enters the collection range, it is considered to agree to collect the personal information. Or, on the device for processing personal information, through the pop-up information or by uploading the personal information by the person himself, the personal authorization is obtained under the condition that the personal information processing rules are informed by obvious mark / information. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and processing personal information type, etc.

[0080] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A vehicle cleanliness detection method characterized by, The method comprises the following steps: obtaining a vehicle image to be detected; segmenting the vehicle image to be detected to obtain a plurality of vehicle component images, wherein the vehicle component images are images in which key components of a vehicle in the vehicle image to be detected are located; extracting features of the vehicle component images by using a feature extraction model in a cleanliness recognition model to obtain global features and local features, wherein the global features are semantic features of the vehicle in the vehicle component images, and the local features are detailed features of local regions in the vehicle component images; fusing the global features and the local features by using a cleanliness prediction model in the cleanliness recognition model to obtain fused features, and predicting the fused features to obtain a cleanliness score of each vehicle component image; wherein the training of the cleanliness recognition model comprises: obtaining a vehicle cleanliness training set, wherein the vehicle cleanliness training set comprises vehicle component sample images and label values corresponding to the vehicle component sample images; processing the vehicle component sample images by using the cleanliness recognition model to obtain cleanliness prediction scores; calculating the cleanliness prediction scores and the corresponding label values to obtain a training loss; based on the training loss, determining whether the cleanliness recognition model meets a training end condition, if not, adjusting the model parameters of the cleanliness recognition model, and returning to the step of processing the vehicle component sample images by using the cleanliness recognition model until the cleanliness recognition model meets the training end condition.

2. The vehicle cleanliness detection method according to claim 1, characterized by, The feature extraction model comprises a plurality of convolution layers connected in sequence, and the step of extracting features of the vehicle component images by using the feature extraction model in the cleanliness recognition model to obtain global features and local features comprises: extracting features of the vehicle component images by using the plurality of convolution layers connected in sequence to obtain the global features; determining feature maps output by part of the convolution layers as the local features.

3. The vehicle cleanliness detection method according to claim 1, characterized by, The cleanliness prediction model comprises a feature splicing layer and a plurality of fully connected layers connected in sequence, and the step of fusing the global features and the local features by using the cleanliness prediction model in the cleanliness recognition model to obtain fused features comprises: fusing the global features and the local features by using the feature splicing layer to obtain the fused features; predicting the fused features by using the plurality of fully connected layers connected in sequence to obtain the cleanliness score.

4. The vehicle cleanliness detection method according to claim 1, characterized by, The sizes of all the vehicle component sample images are preset sizes, and the step of obtaining a vehicle cleanliness training set comprises: obtaining an original sample image and determining whether the size of the original sample image is the preset size, wherein the preset size comprises a preset length and a preset width; if yes, determining the original sample image as the vehicle component sample image; If not, a ratio of a length of the original sample image to the preset length is calculated to obtain a first ratio, and a ratio of a width of the original sample image to the preset width is calculated to obtain a second ratio; based on the first ratio and the second ratio, the original sample image is processed to generate the vehicle component sample image.

5. The vehicle cleanliness detection method according to claim 4, characterized by, The step of processing the original sample image based on the first ratio and the second ratio to generate the vehicle component sample image comprises: when the first ratio is equal to the second ratio, the first ratio is determined as a scaling ratio, and the original sample image is scaled according to the scaling ratio to obtain the vehicle component sample image; when the first ratio is less than the second ratio, the first ratio is determined as a scaling ratio, the original sample image is scaled according to the scaling ratio to obtain a first scaled sample image, and the first scaled sample image is filled to obtain the vehicle component sample image; when the first ratio is greater than the second ratio, the second ratio is determined as a scaling ratio, the original sample image is scaled according to the scaling ratio to obtain a second scaled sample image, and the second scaled sample image is filled to obtain the vehicle component sample image.

6. The vehicle cleanliness detection method according to claim 1, characterized by, The step of obtaining the vehicle image to be detected comprises: obtaining an original vehicle image, and detecting the original vehicle image by using a vehicle type recognition model to obtain the type of vehicle in the original vehicle image; judging whether the type of vehicle is a preset vehicle type; if yes, determining that the original vehicle image is the vehicle image to be detected; if not, returning to the step of obtaining the original vehicle image until the vehicle image to be detected is generated.

7. The vehicle cleanliness detection method according to claim 6, characterized by The vehicle type recognition model comprises a data input layer, a feature extraction layer, and a classification layer, and the feature extraction layer comprises a sub-feature extraction module and an attention module. The step of detecting the original vehicle image by using the vehicle type recognition model to obtain the type of vehicle in the original vehicle image comprises: inputting the original vehicle image into the sub-feature extraction module by using the data input layer; extracting features of the original vehicle image by using the sub-feature extraction module to obtain original image features; enhancing the original image features by using the attention module to obtain enhanced features; classifying the enhanced features by using the classification layer to obtain the type of vehicle.

8. The vehicle cleanliness detection method according to claim 1, characterized by, The method further comprises: weighting and summing all the cleanliness scores to obtain an overall cleanliness score; judging whether the overall cleanliness score exceeds a preset threshold; if yes, determining that the vehicle does not need to be cleaned; if not, determining that the vehicle needs to be cleaned.

9. The vehicle cleanliness detection method according to claim 8, characterized by, The step of weighting and summing all the cleanliness scores to obtain an overall cleanliness score comprises: obtaining a preset weight mapping table, wherein the preset weight mapping table comprises a corresponding relationship between the types of the key components and weight values; matching the types of the key components with the preset weight mapping table to obtain the weight values corresponding to the vehicle component images; and Based on the weight value corresponding to the vehicle component image, a plurality of cleanliness scores are weighted and summed to obtain the overall cleanliness score.

10. The vehicle cleanliness detection method according to claim 1, characterized by, The step of segmenting the vehicle image to be detected to obtain a plurality of vehicle component images comprises: Using a vehicle key component detection model to detect the vehicle image to be detected to obtain key component information; Based on the key component information, the vehicle image to be detected is segmented into the plurality of vehicle component images.

11. The vehicle cleanliness detection method according to claim 10, characterized by, The vehicle key component detection model comprises a backbone network and a detection network; the step of using the vehicle key component detection model to detect the vehicle image to be detected to obtain key component information comprises: Using the backbone network to extract features of the vehicle image to be detected to obtain image feature information; Using the detection network to detect the image feature information to obtain the key component information.

12. A vehicle cleanliness detection apparatus characterized by comprising: A computer program product comprising a memory and a processor connected to each other, wherein the memory is configured to store a computer program, and the computer program, when executed by the processor, is configured to implement the vehicle cleanliness detection method according to any one of claims 1-11.

13. A computer readable storage medium for storing a computer program, characterized in that, The computer program, when executed by the processor, is configured to implement the vehicle cleanliness detection method according to any one of claims 1-11.

Images