Vehicle transportation status detection method, device, electronic device and storage medium

Abstract

The present invention provides a vehicle transport state detection method, device, electronic device and storage medium, wherein the method comprises: obtaining a cargo box image of a vehicle to be detected; inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model; wherein the transport state detection model is obtained by training based on sample cargo box images and sample transport states corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migration sample cargo box images; the migration sample cargo box images are obtained by style migration of the normal sample cargo box images based on the abnormal sample cargo box images. The method, device, electronic device and storage medium provided by the present invention improve the diversity of samples, improve the robustness of the transport state detection model, and improve the effect of vehicle transport state detection.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a vehicle transportation status detection method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of urban construction, the demand for muck trucks has increased rapidly. During the transportation of muck trucks, due to poor covering or illegal dumping by drivers, the muck is spilled and leaked during driving, seriously polluting the urban environment and increasing the difficulty of cleaning. Therefore, it is necessary to monitor the transportation status of muck trucks in real time and manage them in a standardized manner.

[0003] In the prior art, cameras are installed on muck trucks to identify the empty, overloaded, sealed, or lifted state of the cargo box, as well as the type of muck medium, and to realize video monitoring of the cargo box, so as to reduce the spillage and dripping problems caused by the muck truck not being sealed or overloaded, and to a certain extent reduce the dust pollution on the road. However, muck trucks mostly operate at night, and the lighting environment at night is complex, with backlighting and strong direct light. At the same time, dust from the construction site sometimes covers the camera lens, resulting in poor image quality captured by the camera, making the detection effect of the transportation status of the muck truck poor. Summary of the invention

[0004] The present invention provides a vehicle transportation state detection method, device, electronic equipment and storage medium, which are used to solve the technical problem that the vehicle transportation state detection method in the prior art has poor effect.

[0005] The present invention provides a vehicle transportation status detection method, comprising:

[0006] Acquire a cargo box image of the vehicle to be detected;

[0007] Inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model;

[0008] Among them, the transport status detection model is obtained by training based on sample cargo box images and sample transport status corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by style transfer of the normal sample cargo box images based on the abnormal sample cargo box images.

[0009] According to the vehicle transport state detection method provided by the present invention, the transport state detection model is trained based on the following steps:

[0010] Alternately training the initial model based on the first sample and the second sample to obtain the transportation state detection model;

[0011] Among them, the first sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, the second sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, an abnormal sample cargo box image and a sample transportation status corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and a sample transportation status corresponding to the migration sample cargo box image.

[0012] According to the vehicle transport state detection method provided by the present invention, the initial model is alternately trained based on the first sample and the second sample, comprising:

[0013] When the first sample is used to train the initial model, a generative adversarial network composed of the initial model and the generator is trained based on the normal sample cargo box image and the sample transportation state corresponding to the normal sample cargo box image;

[0014] The generator is used to generate fake samples based on given noise, and the initial model is used to perform authenticity judgment and transportation status detection on the input sample images, and the input sample images include the normal sample cargo box images and the fake samples.

[0015] According to the vehicle transport state detection method provided by the present invention, the step of inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model comprises:

[0016] Inputting the cargo box image into the feature extraction layer of the transport state detection model to obtain cargo box image features output by the feature extraction layer;

[0017] Inputting the cargo box image features into the coordinate attention layer of the transport state detection model to obtain the image position enhancement features output by the coordinate attention layer;

[0018] Inputting the cargo box image into the residual connection layer of the transportation state detection model to obtain the image representation features output by the residual connection layer;

[0019] The image position enhancement feature and the image representation feature are input into the feature fusion layer of the transport state detection model to obtain the transport state of the vehicle to be detected output by the feature fusion layer.

[0020] According to the vehicle transport status detection method provided by the present invention, the migration sample cargo box image is determined based on the following steps:

[0021] Inputting the abnormal sample cargo box image and the normal sample cargo box image into an image style transfer model to obtain a transfer sample cargo box image output by the image style transfer model;

[0022] The image style transfer model includes a feature encoding layer, an adaptive instance regularization layer and a feature decoding layer connected in sequence; the loss of the image style transfer model includes consistency loss, content loss and / or style loss.

[0023] According to the vehicle transportation state detection method provided by the present invention, the consistency loss of the image style transfer model is determined based on the following steps:

[0024] Input any one of the images as an abnormal sample cargo box image and a normal sample cargo box image to the image style transfer model at the same time, and obtain a transfer image corresponding to any one of the images output by the image style transfer model;

[0025] Based on the any image and a transfer image corresponding to the any image, a consistency loss of the image style transfer model is determined.

[0026] According to the vehicle transportation state detection method provided by the present invention, the content loss and style loss of the image style transfer model are determined based on the following steps:

[0027] Inputting the migration sample cargo box image and the feature map output by the adaptive instance regularization layer into the feature encoding layer to determine the content loss of the image style migration model;

[0028] The abnormal sample cargo box image and the migration sample cargo box image are input into the feature encoding layer to determine the style loss of the image style migration model.

[0029] The present invention provides a vehicle transportation state detection device, comprising:

[0030] An acquisition unit, used for acquiring a cargo box image of a vehicle to be detected;

[0031] A detection unit, used for inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model;

[0032] Among them, the transport status detection model is obtained by training based on sample cargo box images and sample transport status corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by style transfer of the normal sample cargo box images based on the abnormal sample cargo box images.

[0033] The present invention provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the vehicle transport status detection method when executing the program.

[0034] The present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle transportation status detection method are implemented.

[0035] The vehicle transport state detection method, device, electronic device and storage medium provided by the present invention can determine the transport state of a vehicle to be detected based on the cargo box image of the vehicle to be detected by training a transport state detection model using sample cargo box images and sample transport states corresponding to the sample cargo box images. The migrated sample cargo box images obtained by style migration effectively expand the number and types of sample cargo box images and improve the diversity of samples. The initial model is trained using normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images, thereby improving the learning ability of the transport state detection model and the robustness of the transport state detection model. The obtained transport state detection model has strong adaptability to cargo box images with poor image quality obtained in the actual detection process, can detect the accurate transport state of the vehicle to be detected, and improves the effect of vehicle transport state detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a process flow of a vehicle transport status detection method provided by the present invention;

[0038] Figure 2 A schematic diagram of the structure of a generative adversarial network provided by the present invention;

[0039] Figure 3 A network structure diagram of the transport status detection model provided by the present invention;

[0040] Figure 4 A schematic diagram of the structure of the image style transfer model provided by the present invention;

[0041] Figure 5 A schematic diagram of the structure of a vehicle transport status detection device provided by the present invention;

[0042] Figure 6 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Figure 1 A flow chart of the vehicle transport status detection method provided by the present invention is shown as follows: Figure 1 As shown, the method includes:

[0045] Step 110, obtaining an image of a cargo box of a vehicle to be detected.

[0046] Specifically, the vehicle to be detected may be a vehicle with a cargo box, such as a muck truck or a mining truck. The cargo box image is an image obtained by capturing the cargo box of the vehicle. The cargo box of the vehicle to be detected may be monitored by installing a camera on the front of the vehicle to be detected or above the front of the cargo box.

[0047] Step 120, input the cargo box image into the transport status detection model to obtain the transport status of the vehicle to be detected output by the transport status detection model; wherein the transport status detection model is trained based on the sample cargo box image and the sample transport status corresponding to the sample cargo box image; the sample cargo box image includes a normal sample cargo box image, an abnormal sample cargo box image and a migrated sample cargo box image; the migrated sample cargo box image is obtained by performing style transfer on the normal sample cargo box image based on the abnormal sample cargo box image.

[0048] Specifically, the transport state of the vehicle to be detected refers to the state of the cargo box carrying the cargo. The transport state can be set as needed, for example, the transport state can include empty and full, and can also include tarpaulin closed and tarpaulin opened.

[0049] The transport status detection model can be used to detect and identify the cargo box image to obtain the transport status of the vehicle to be detected. The transport status detection model is used to extract features from the cargo box image, and learn from the extracted features to obtain information such as the state of the objects in the cargo box in the image, thereby determining the transport status of the vehicle to be detected.

[0050] The transport status detection model can be pre-trained, and specifically can be obtained through the following training method: first, a large number of sample cargo box images are collected; second, each sample cargo box image is labeled to determine the sample transport status corresponding to each sample cargo box image; then, based on a large number of sample cargo box images and the sample transport status corresponding to each sample cargo box image, the initial model is trained to improve the initial model's prediction ability for the transport status corresponding to the cargo box image, thereby obtaining the transport status detection model.

[0051] For the vehicle to be detected, the viewing angle of the camera installed on the vehicle body is relatively fixed. Therefore, the scene in which the cargo box image is collected changes little, which is prone to overfitting when used for training the model. In addition, since the vehicle may operate for a long time at night, the camera faces various complex lighting environments such as no light, backlight, or even strong light, and may also face situations such as dust occlusion, so the quality of the actual collected cargo box images is poor. Therefore, the robustness of the transportation status detection model can be improved through a variety of sample training methods.

[0052] The initial model can be trained by normal sample container images, abnormal sample container images and migration sample container images to obtain a transportation status detection model. Among them, the normal sample container images are container images taken under good exposure conditions such as sufficient light, and the abnormal sample container images are container images taken under poor exposure conditions such as dust occlusion, backlighting or strong direct light. Normal sample container images and abnormal sample container images can be distinguished according to the pixel value of each pixel in the image. For example, the pixel value of each pixel in the image can be weighted and averaged. If the pixel average value is greater than the preset brightness threshold or less than the preset darkness threshold, the image is considered to be an abnormal sample container image. If the pixel average value is between the preset darkness threshold and the preset brightness threshold, the image is considered to be a normal sample container image. Among them, the preset darkness threshold can be 60, and the preset brightness threshold can be 200.

[0053] In the process of collecting cargo box images, the number of normal sample cargo box images accounts for the majority, and the number of abnormal sample cargo box images is relatively small. Training the initial model with fewer abnormal sample cargo box images will lead to poor model robustness. At this time, the style transfer method can be used to expand the samples, which can be obtained by performing style transfer on the normal sample cargo box images based on the abnormal sample cargo box images.

[0054] The vehicle transport status detection method provided by the embodiment of the present invention can determine the transport status of the vehicle to be detected based on the cargo box image of the vehicle to be detected through the transport status detection model obtained by training with the sample cargo box images and the sample transport status corresponding to the sample cargo box images. The migration sample cargo box images obtained by style migration effectively expand the number and types of sample cargo box images and improve the diversity of samples. The initial model is trained with the normal sample cargo box images, the abnormal sample cargo box images and the migration sample cargo box images, which improves the learning ability of the transport status detection model and the robustness of the transport status detection model. The obtained transport status detection model has strong adaptability to the cargo box images with poor image quality obtained in the actual detection process, can detect the accurate transport status of the vehicle to be detected, and improves the effect of vehicle transport status detection.

[0055] Based on the above embodiment, the transport status detection model is trained based on the following steps:

[0056] The initial model is alternately trained based on the first sample and the second sample to obtain a transportation state detection model;

[0057] Among them, the first sample is composed of a normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image, the second sample is composed of a normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image, an abnormal sample cargo box image and the sample transportation status corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and the sample transportation status corresponding to the migration sample cargo box image.

[0058] Specifically, an alternating training method may be used to train the initial model to obtain a transport state detection model. Normal sample cargo box images, abnormal sample cargo box images, and migration sample cargo box images may be classified into first samples and second samples.

[0059] The first sample consists of a normal sample cargo box image and a sample transportation state corresponding to the normal sample cargo box image. In most cases, the collected cargo box images meet good lighting conditions or are free of dust. Therefore, the first sample is used to train the initial model's recognition ability for normal sample cargo box images.

[0060] The second sample is composed of a normal sample cargo box image and a sample transportation state corresponding to the normal sample cargo box image, an abnormal sample cargo box image and a sample transportation state corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and a sample transportation state corresponding to the migration sample cargo box image. In the case of poor lighting conditions or occlusion, the second sample can be used to train the initial model's recognition ability for a variety of sample cargo box images. For example, three samples can be extracted from the second sample to train the initial model.

[0061] During alternating training, the first sample and the second sample can be selected according to the set number of samples, and then the initial model is trained alternately until a preset number of training steps is reached or the loss of the initial model reaches a set threshold, thereby obtaining a transportation status detection model.

[0062] Based on any of the above embodiments, the initial model is alternately trained based on the first sample and the second sample, including:

[0063] When the first sample is used to train the initial model, a generative adversarial network composed of the initial model and the generator is trained based on the normal sample cargo box image and the sample transportation state corresponding to the normal sample cargo box image;

[0064] The generator is used to generate fake samples based on given noise, and the initial model is used to judge the authenticity and transportation status of the input sample images. The input sample images include normal sample cargo box images and fake samples.

[0065] Specifically, since the perspective of the cargo box image is fixed, normal sample cargo box images are mostly collected under good lighting conditions, and the data variance is small, which can easily lead to overfitting of the initial model. Therefore, generative adversarial training can be used for the initial model.

[0066] When the first sample is used to train the initial model, a generative adversarial network is constructed using the initial model as a discriminator, and the type of the generative adversarial network can be a WGAN (Wasserstein GAN) structure. The given noise can be Gaussian noise.

[0067] Figure 2 A schematic diagram of the structure of the generative adversarial network provided by the present invention is shown in FIG. Figure 2 As shown, the discriminator D is the initial model. The principle of generative adversarial training is: the generator G inputs Gaussian noise Z (noise) to generate a fake image X (fake). The fake image X (fake) and the real image X (real) (data) are input to the discriminator D for discrimination, and the probability of the input image belonging to a real sample (real) or a fake sample (fake) and the image category (cls1, cls2, ...) are obtained.

[0068] According to the above principle, the given noise is input into the generator of the generative adversarial network, and a large number of fake samples are generated by the generator. The discriminator is used to judge the authenticity and transportation status of the input sample image. According to the normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image, as well as the fake sample, the discriminator is trained so that the discriminator can not only identify the true and false probability of the sample, but also identify the classification label corresponding to the true sample (normal sample cargo box image), which can improve the discrimination ability of the transportation status detection model, that is, the learning ability of the transportation status detection model for image features.

[0069] Given noise, the generator of the generative adversarial network can simulate the data distribution of actual cargo box images. When it is impossible to obtain sample cargo box images under various acquisition conditions, the use of the generative adversarial network can simulate the generation of samples with different data distributions, solving the problem of model overfitting that may be caused by small sample data and fixed scenes, improving the stability of the transportation status detection model and improving the generalization ability of the transportation status detection model.

[0070] Based on any of the above embodiments, step 120 includes:

[0071] Input the cargo box image into the feature extraction layer of the transportation state detection model to obtain the cargo box image features output by the feature extraction layer;

[0072] Input the cargo box image features into the coordinate attention layer of the transportation status detection model to obtain the image position enhancement features output by the coordinate attention layer;

[0073] Input the cargo box image into the residual connection layer of the transportation status detection model to obtain the image representation features output by the residual connection layer;

[0074] The image position enhancement features and the image representation features are input into the feature fusion layer of the transport state detection model to obtain the transport state of the vehicle to be detected output by the feature fusion layer.

[0075] Specifically, a coordinate attention mechanism and a residual connection can be added to the model structure of the transportation state detection model. Accordingly, from the network structure, the transportation state detection model can include a feature extraction layer, a coordinate attention layer, a residual connection layer, and a feature fusion layer.

[0076] Figure 3 The network structure diagram of the transport status detection model provided by the present invention is as follows: Figure 3As shown in the figure, the cargo box image is input into the feature extraction layer of the transport status detection model, and the cargo box image features output by the feature extraction layer can be obtained. The feature extraction layer can adopt the MobileNet V2 structure. MobileNet V2 is a lightweight convolutional neural network, which can include a 1x1 convolution activation (ReLU6) layer, a 3x3 deep convolution activation (ReLU6) layer and a 1x1 convolution linear (Linear) layer connected in sequence. MobileNet V2 has an inverted residual (Inverted Residuals) and linear bottleneck (Linear Bottlenecks) structure, which can reduce the number of parameters and calculations while improving the feature extraction function.

[0077] The cargo box image features are input into the coordinate attention layer of the transportation status detection model to obtain the image position enhancement features output by the coordinate attention layer. The coordinate attention layer adopts the coordinate attention mechanism, which can embed the position information into the channel attention, so that the lightweight network can pay attention to a larger area while avoiding a large amount of computational overhead. Using two one-dimensional global pooling operations, the input features in the horizontal direction (X direction) and the vertical direction (Y direction) are aggregated into two independent direction-aware feature maps, and then these two feature maps embedded with specific direction information are encoded into two attention maps, each of which captures the long-range dependency of the input feature map along a spatial direction. Therefore, the position information is saved in the generated attention map. The two attention maps are then multiplied by the input feature map to enhance the representation ability of the feature map.

[0078] The cargo box image is input into the residual connection layer of the transportation status detection model. The residual connection layer uses a 1x1 convolutional linear layer to extract the image representation features of the cargo box image.

[0079] Finally, the image position enhancement feature and the image representation feature are input into the feature fusion layer of the transport status detection model. The feature fusion layer fuses these two features and identifies them to obtain the transport status of the vehicle to be detected.

[0080] In addition, when the first sample is used to train the transport status detection model, its generator can also adopt the same network structure as the transport status detection model, that is, the network structure of MobileNet V2 with added coordinate attention mechanism and residual connection.

[0081] The vehicle transport status detection method provided by the embodiment of the present invention adds a coordinate attention mechanism and a residual connection to the network structure of the transport status detection model, thereby improving the sensitivity of the transport status detection model to information such as direction and position, and can pay more attention to the detail information in the cargo box image, thereby abstracting the overall category information of the image, thereby improving the learning ability and accuracy of the transport status detection model.

[0082] Based on any of the above embodiments, the migration sample container image is determined based on the following steps:

[0083] The abnormal sample cargo box images and the normal sample cargo box images are input into the image style transfer model to obtain the transfer sample cargo box images output by the image style transfer model;

[0084] The image style transfer model includes a feature encoding layer, an adaptive instance regularization layer and a feature decoding layer connected in sequence; the loss of the image style transfer model includes consistency loss, content loss and / or style loss.

[0085] Specifically, the image style transfer model includes a feature encoding layer, an adaptive instance normalization layer, and a feature decoding layer connected in sequence. Among them, the adaptive instance normalization layer adopts the AdaIN (adaptive instance normalization) structure, and adjusts the pixel mean and standard deviation of the corresponding channel in the normal sample container image according to the pixel mean and standard deviation of each channel in the abnormal sample container image, so as to realize the style transfer of the abnormal sample container image to the normal sample container image, which can be expressed as follows:

[0086]

[0087] Where x is the value in any channel of the normal sample container image, y is the value in the corresponding channel of the abnormal sample container image, μ(x) is the mean of x, μ(y) is the mean of y, σ(x) is the standard deviation of x, and σ(y) is the standard deviation of y.

[0088] When training an image style transfer model, you can use identity loss, content loss, and style loss.

[0089] Consistent loss is used to calculate the loss between the output image and the input image of the image style transfer model when the abnormal sample cargo box image and the normal sample cargo box image are the same image. This can prevent the image style transfer model from having large deviations during the training process.

[0090] Content loss is used to calculate the loss between the transferred sample container image and the normal sample container image, which can enable the image style transfer model to maintain as much detail as possible during the training process.

[0091] Style loss is used to calculate the loss between the transferred sample cargo box image and the abnormal sample cargo box image, which can enable the image style transfer model to transfer as many style features as possible during the training process.

[0092] The vehicle transport status detection method provided by the embodiment of the present invention transfers the style of the abnormal sample cargo box image to the normal sample cargo box image through the image style transfer model to obtain the transferred sample cargo box image, thereby achieving the expansion of the abnormal sample cargo box image and improving the data diversity of the sample.

[0093] Based on any of the above embodiments, the consistency loss of the image style transfer model is determined based on the following steps:

[0094] Input any image as both an abnormal sample cargo box image and a normal sample cargo box image into the image style transfer model, and obtain a transfer image corresponding to any image output by the image style transfer model;

[0095] Determine the consistency loss of the image style transfer model based on any image and its corresponding transfer image.

[0096] Specifically, any image can be simultaneously used as an abnormal sample cargo box image and a normal sample cargo box image and input into the image style transfer model to obtain a transfer image corresponding to the image output by the image style transfer model. Any image can be selected from existing cargo box images.

[0097] The consistent loss can be calculated based on the L2 norm loss of the two images. For example, the L2 norm loss of the image is determined based on the pixel value of the pixel point in the image, and the difference between the L2 norm losses of the two images is used as the consistent loss.

[0098] In addition, the consistency loss can be determined based on the Euclidean distance between corresponding pixels in two images.

[0099] Based on any of the above embodiments, the content loss and style loss of the image style transfer model are determined based on the following steps:

[0100] The feature map output by the adaptive instance regularization layer and the transfer sample container image are input into the feature encoding layer to determine the content loss of the image style transfer model;

[0101] The abnormal sample cargo box images and the transferred sample cargo box images are input into the feature encoding layer to determine the style loss of the image style transfer model.

[0102] Specifically, Figure 4A schematic diagram of the structure of the image style transfer model provided by the present invention, such as Figure 4 As shown, the model includes a feature encoding layer (Encoder), an adaptive instance regularization layer (AdaIN) and a feature decoding layer (Decoder).

[0103] The feature map output by the adaptive instance regularization layer is the feature map after style transfer. The feature map and the transfer sample container image are input into the feature encoding layer to obtain the content loss of the image style transfer model.

[0104] The abnormal sample cargo box images and the transferred sample cargo box images are input into the feature encoding layer to determine the style loss of the image style transfer model.

[0105] Based on any of the above embodiments, the initial model is alternately trained based on the first sample and the second sample to obtain the transport state detection model, which includes:

[0106] The sample cargo box images are preprocessed, including random rotation, random cropping, random flipping and random erasing.

[0107] Specifically, preprocessing can also be used to preprocess the sample cargo box image. Random rotation refers to rotating the sample cargo box image according to a randomly generated angle. Random cropping refers to randomly selecting a sample cargo box image to crop and adjust its size. Random flipping refers to horizontally flipping a randomly selected sample cargo box image. Random erasing refers to randomly using multiple boxes, such as a bound box and an anchor box, to erase part of the content in the sample cargo box image.

[0108] Based on any of the above embodiments, Figure 5 The schematic diagram of the structure of the vehicle transport status detection device provided by the present invention is as follows: Figure 5 As shown, the device comprises:

[0109] An acquisition unit 510 is used to acquire a cargo box image of a vehicle to be detected;

[0110] The detection unit 520 is used to input the cargo box image into the transportation state detection model to obtain the transportation state of the vehicle to be detected output by the transportation state detection model;

[0111] Among them, the transportation status detection model is trained based on sample cargo box images and sample transportation status corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by style transfer of normal sample cargo box images based on abnormal sample cargo box images.

[0112] The vehicle transport status detection device provided by the embodiment of the present invention can determine the transport status of the vehicle to be detected based on the cargo box image of the vehicle to be detected through the transport status detection model obtained by training the sample cargo box images and the sample transport status corresponding to the sample cargo box images. The migration sample cargo box images obtained by style migration effectively expand the number and types of sample cargo box images and improve the diversity of samples. The initial model is trained by normal sample cargo box images, abnormal sample cargo box images and migration sample cargo box images, which improves the learning ability of the transport status detection model. The obtained transport status detection model has strong adaptability to cargo box images with poor image quality obtained in the actual detection process, can detect the accurate transport status of the vehicle to be detected, and improves the robustness of the transport status detection model.

[0113] Based on any of the above embodiments, it also includes:

[0114] A training unit, used for alternately training the initial model based on the first sample and the second sample to obtain a transportation status detection model;

[0115] Among them, the first sample is composed of a normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image, the second sample is composed of a normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image, an abnormal sample cargo box image and the sample transportation status corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and the sample transportation status corresponding to the migration sample cargo box image.

[0116] Based on any of the above embodiments, the training unit includes:

[0117] The generative adversarial subunit is used to train the generative adversarial network composed of the initial model and the generator based on the normal sample cargo box image and the sample transportation status corresponding to the normal sample cargo box image when the first sample is used to train the initial model; the generator is used to generate false samples based on given noise, and the initial model is used to judge the authenticity and transportation status of the input sample images, and the input sample images include normal sample cargo box images and false samples.

[0118] Based on any of the above embodiments, the detection unit is used for:

[0119] Input the cargo box image into the feature extraction layer of the transportation state detection model to obtain the cargo box image features output by the feature extraction layer;

[0120] Input the cargo box image features into the coordinate attention layer of the transportation status detection model to obtain the image position enhancement features output by the coordinate attention layer;

[0121] Input the cargo box image into the residual connection layer of the transportation status detection model to obtain the image representation features output by the residual connection layer;

[0122] The image position enhancement features and the image representation features are input into the feature fusion layer of the transport state detection model to obtain the transport state of the vehicle to be detected output by the feature fusion layer.

[0123] Based on any of the above embodiments, it also includes:

[0124] A migration unit, used to input abnormal sample cargo box images and normal sample cargo box images into the image style migration model to obtain a migration sample cargo box image output by the image style migration model;

[0125] The image style transfer model includes a feature encoding layer, an adaptive instance regularization layer and a feature decoding layer connected in sequence; the loss of the image style transfer model includes consistency loss, content loss and / or style loss.

[0126] Based on any of the above embodiments, the migration unit includes:

[0127] The consistent loss determination subunit is used to input any image as an abnormal sample cargo box image and a normal sample cargo box image into the image style transfer model at the same time, and obtain a migration image corresponding to any image output by the image style transfer model; based on any image and the migration image corresponding to any image, determine the consistent loss of the image style transfer model.

[0128] Based on any of the above embodiments, the migration unit includes:

[0129] A content loss determination subunit is used to input the migration sample cargo box image and the feature map output by the adaptive instance regularization layer into the feature encoding layer to determine the content loss of the image style migration model;

[0130] The style loss determination subunit is used to input the abnormal sample cargo box image and the migration sample cargo box image into the feature encoding layer to determine the style loss of the image style migration model.

[0131] Based on any of the above embodiments, Figure 6 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor (Processor) 610, a communication interface (Communications Interface) 620, a memory (Memory) 630 and a communication bus (Communications Bus) 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic command in the memory 630 to execute the following method:

[0132] Obtain a cargo box image of a vehicle to be detected; input the cargo box image into a transport status detection model to obtain the transport status of the vehicle to be detected output by the transport status detection model; wherein the transport status detection model is trained based on sample cargo box images and sample transport statuses corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by performing style transfer on normal sample cargo box images based on abnormal sample cargo box images.

[0133] In addition, the logic commands in the above-mentioned memory 630 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0134] The processor in the electronic device provided in the embodiment of the present invention can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.

[0135] An embodiment of the present invention further provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including:

[0136] Obtain a cargo box image of a vehicle to be detected; input the cargo box image into a transport status detection model to obtain the transport status of the vehicle to be detected output by the transport status detection model; wherein the transport status detection model is trained based on sample cargo box images and sample transport statuses corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by performing style transfer on normal sample cargo box images based on abnormal sample cargo box images.

[0137] When the computer program stored on the non-transitory computer-readable storage medium provided by the embodiment of the present invention is executed, the above method is implemented. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.

[0138] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle transportation status detection method, characterized in that: include: Obtaining a cargo box image of the vehicle to be detected; Inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model; The transport state detection model is obtained by training based on sample cargo box images and sample transport states corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by performing style migration on the normal sample cargo box images based on the abnormal sample cargo box images; The transport status detection model is trained based on the following steps: Alternately training the initial model based on the first sample and the second sample to obtain the transportation state detection model; The first sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, the second sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, an abnormal sample cargo box image and a sample transportation status corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and a sample transportation status corresponding to the migration sample cargo box image; Determine a pixel average based on the pixel value of each pixel in any container image; If the pixel average value is greater than the preset brightness threshold or less than the preset darkness threshold, then any of the cargo box images is an abnormal sample cargo box image; if the pixel average value is between the preset darkness threshold and the preset brightness threshold, then any of the cargo box images is a normal sample cargo box image.

2. The vehicle transportation status detection method according to claim 1, characterized in that: The alternately training the initial model based on the first sample and the second sample comprises: When the first sample is used to train the initial model, a generative adversarial network composed of the initial model and the generator is trained based on the normal sample cargo box image and the sample transportation state corresponding to the normal sample cargo box image; The generator is used to generate fake samples based on given noise, and the initial model is used to perform authenticity judgment and transportation status detection on the input sample images, and the input sample images include the normal sample cargo box images and the fake samples.

3. The vehicle transportation status detection method according to claim 1, characterized in that: The step of inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model includes: Inputting the cargo box image into the feature extraction layer of the transport state detection model to obtain cargo box image features output by the feature extraction layer; Inputting the cargo box image features into the coordinate attention layer of the transport state detection model to obtain the image position enhancement features output by the coordinate attention layer; Inputting the cargo box image into the residual connection layer of the transportation state detection model to obtain the image representation features output by the residual connection layer; The image position enhancement feature and the image representation feature are input into the feature fusion layer of the transport state detection model to obtain the transport state of the vehicle to be detected output by the feature fusion layer.

4. The vehicle transportation status detection method according to any one of claims 1 to 3, characterized in that: The migration sample container image is determined based on the following steps: Inputting the abnormal sample cargo box image and the normal sample cargo box image into an image style transfer model to obtain a transfer sample cargo box image output by the image style transfer model; The image style transfer model includes a feature encoding layer, an adaptive instance regularization layer and a feature decoding layer connected in sequence; the loss of the image style transfer model includes consistency loss, content loss and / or style loss.

5. The vehicle transportation status detection method according to claim 4, characterized in that: The consistency loss of the image style transfer model is determined based on the following steps: Input any image as an abnormal sample cargo box image and a normal sample cargo box image to the image style transfer model at the same time, and obtain a transfer image corresponding to the any image output by the image style transfer model; Based on the any image and a transfer image corresponding to the any image, a consistency loss of the image style transfer model is determined.

6. The vehicle transportation status detection method according to claim 4, characterized in that: The content loss and style loss of the image style transfer model are determined based on the following steps: Inputting the migration sample cargo box image and the feature map output by the adaptive instance regularization layer into the feature encoding layer to determine the content loss of the image style migration model; The abnormal sample cargo box image and the migration sample cargo box image are input into the feature encoding layer to determine the style loss of the image style migration model.

7. A vehicle transport status detection device, characterized in that: include: An acquisition unit, used to acquire a cargo box image of a vehicle to be detected; A detection unit, used for inputting the cargo box image into a transport state detection model to obtain the transport state of the vehicle to be detected output by the transport state detection model; The transport state detection model is obtained by training based on sample cargo box images and sample transport states corresponding to the sample cargo box images; the sample cargo box images include normal sample cargo box images, abnormal sample cargo box images and migrated sample cargo box images; the migrated sample cargo box images are obtained by performing style migration on the normal sample cargo box images based on the abnormal sample cargo box images; Also includes: A training unit, configured to perform alternate training on the initial model based on the first sample and the second sample to obtain the transport state detection model; The first sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, the second sample is composed of a normal sample cargo box image and a sample transportation status corresponding to the normal sample cargo box image, an abnormal sample cargo box image and a sample transportation status corresponding to the abnormal sample cargo box image, and a migration sample cargo box image and a sample transportation status corresponding to the migration sample cargo box image; Determine a pixel average based on the pixel value of each pixel in any container image; If the pixel average value is greater than the preset brightness threshold or less than the preset darkness threshold, then any of the cargo box images is an abnormal sample cargo box image; if the pixel average value is between the preset darkness threshold and the preset brightness threshold, then any of the cargo box images is a normal sample cargo box image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the vehicle transportation status detection method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the vehicle transportation status detection method according to any one of claims 1 to 6 are implemented.

Images