Vehicle Loading Picture Extraction Method and System

Abstract

The present invention discloses a method and system for extracting vehicle loading pictures, belonging to the technical field of image recognition. The method includes the steps of: S1, collecting video frame images from the video of the monitored loading checkpoint at a specified time interval and storing them; S2, performing binary classification recognition of whether there is a vehicle in each collected video frame image to obtain a picture status sequence related to the loading checkpoint arranged in the order of image acquisition time; S3, identifying the 0-1 switching point in the picture status sequence as the departure point, and extracting m elements before the departure point and n elements after it, and then forming a matching sequence with the elements arranged in time order with the departure point; S4, aligning the matching sequence with an equal-length departure mode sequence for similarity matching, and after arranging the calculated matching indexes in ascending order, selecting several video frame images corresponding to the matching indexes ranked at the front as candidate vehicle loading pictures. The present invention improves the recognition accuracy of vehicle loading pictures.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a method and system for extracting vehicle loading pictures. Background Art

[0002] In logistics enterprises, the largest cost is the transportation cost. The most direct and effective way to reduce the transportation cost is to improve the loading rate of each vehicle (the ratio of the volume of goods loaded in the carriage to the total volume of the carriage), so as to achieve the purpose of transporting as many goods as possible with as few vehicles as possible. To improve the loading rate, enterprises need to accurately grasp the current loading rate of each vehicle, so as to formulate targeted loading rate improvement plans and measure the final improvement effect.

[0003] In the prior art, an image recognition method is usually used to automatically judge the loading rate of each vehicle, that is, the loading rate of the vehicle is calculated by image recognition according to the loading pictures of the vehicle when the loading is completed. At present, there are mainly the following two ways to obtain vehicle loading pictures:

[0004] 1. After the vehicle loading is completed, a person goes to the scene to take pictures of the vehicle loading image. However, the manual shooting method has poor flexibility, requires a special person to be responsible for this work, wastes labor costs, and the quality of the images taken manually is uneven. Even the pictures taken may not cover the vehicle tail, resulting in the vehicle loading area not being fully captured, directly affecting the subsequent recognition accuracy of the machine for the loading rate;

[0005] 2. The images of vehicle loading are automatically collected through a camera. However, it is difficult for the existing image recognition models to accurately identify the vehicle loading images at the end of loading among many vehicle loading images, and recognition errors are likely to occur, which also affects the subsequent recognition accuracy of the machine for the loading rate. Summary of the Invention

[0006] The present invention aims to accurately extract the vehicle loading pictures at the end of loading from the monitoring video, and provides a method and system for extracting vehicle loading pictures.

[0007] To achieve this purpose, the present invention adopts the following technical solutions:

[0008] Provide a method for extracting vehicle loading pictures, the steps including:

[0009] S1, collect video frame images from the video of the monitoring loading checkpoint at a specified time interval and store them;

[0010] S2. Perform binary classification recognition of whether there is a vehicle in each of the collected video frame images to obtain a picture status sequence associated with the loading bay, arranged in the order of image acquisition time. The elements in the picture status sequence are represented by "0" and "1". The vehicle status where the vehicle appears at the loading bay and the door is not closed is represented by "0", otherwise by "1".

[0011] S3. Identify the 0-1 switching points in the picture status sequence as the departure points, and extract m elements before the departure point and n elements after it, and then form a matching sequence with the departure point, where the elements are arranged in the order of image acquisition time.

[0012] S4. Align the matching sequence with a departure pattern sequence of the same length for similarity matching, and after arranging the calculated matching indices in ascending order, select the video frame images corresponding to the "0" elements among the departure points corresponding to several of the top-ranked matching indices as candidate vehicle loading pictures.

[0013] As a preferred solution of the present invention, the vehicle loading picture extraction method further includes the step of:

[0014] S5. Associate the vehicle loading pictures with the corresponding transportation tasks according to the associated time between the transportation tasks and the loading bay.

[0015] As a preferred solution of the present invention, the specified time interval in step S1 is 30s.

[0016] As a preferred solution of the present invention, in step S3, m = 59.

[0017] As a preferred solution of the present invention, in step S3, n = 4.

[0018] As a preferred solution of the present invention, in step S4, the following formula (1) is used to perform similarity matching between the matching sequence and the departure pattern sequence:

[0019]

[0020] In formula (1), x i represents the value of the i-th element in the matching sequence;

[0021] y i represents the value of the i-th element in the departure pattern sequence;

[0022] f represents the matching index to be solved.

[0023] As a preferred embodiment of the present invention, the number of candidate vehicle loading pictures extracted after performing similarity matching on the picture status sequence satisfies the constraint condition expressed by the following formula (2):

[0024] c ≤ p + 3 Formula (2)

[0025] In Formula (2), c represents the number of candidate vehicle loading pictures extracted after similarity matching of the picture status sequence;

[0026] p represents the number of transportation tasks associated with the same loading bay.

[0027] As a preferred embodiment of the present invention, when the acquisition time interval between the first candidate vehicle loading picture and the second candidate vehicle loading picture extracted in step S4 is less than the preset time interval, the first acquired vehicle loading picture is retained and the second acquired vehicle loading picture is excluded.

[0028] As a preferred embodiment of the present invention, the preset time interval is 1800s.

[0029] The present invention also provides a vehicle loading picture extraction system that can implement the vehicle loading picture extraction method described above. The system includes:

[0030] An image acquisition module for acquiring video frame images from the video monitoring the loading bay at a specified time interval and storing them in a picture database;

[0031] A vehicle status recognition module connected to the image acquisition module for performing binary classification recognition of whether there is a vehicle in each acquired video frame image to obtain a picture status sequence arranged in the order of image acquisition time associated with the loading bay. The elements in the picture status sequence are represented by "0" and "1". The vehicle status where the vehicle appears at the loading bay and the door is not closed is represented by "0", otherwise by "1";

[0032] A departure point recognition module connected to the vehicle status recognition module for identifying the 0-1 switching point in the picture status sequence as the departure point;

[0033] A matching sequence generation module connected to the departure point recognition module and the vehicle status recognition module respectively for extracting m elements before the departure point and n elements after it and forming a matching sequence with the departure point in the order of image acquisition time of the elements;

[0034] A similarity matching module connected to the matching sequence generation module for aligning the matching sequence with a departure mode sequence of the same length and performing similarity matching to obtain the matching index of the matching sequence and the departure mode sequence;

[0035] A candidate picture extraction module, which is respectively connected to the image acquisition module and the similarity matching module, is used for arranging the calculated matching indexes in ascending order, and then selecting the video frame images corresponding to the "0" elements in the departure points corresponding to several matching indexes ranked in the front as candidate vehicle loading pictures;

[0036] A picture association module, which is connected to the candidate picture extraction module, is used for associating the vehicle loading pictures with the corresponding transportation tasks according to the association time between the transportation tasks and the loading bayonet.

[0037] In the present invention, the video frame sequence collected from the video of the monitored loading bayonet is converted into a picture state sequence, and the departure point is identified in the picture state sequence, and then a matching sequence containing the departure point is formed, and the matching sequence is subjected to similarity matching with the departure mode sequence, and after arranging the matching indexes in ascending order according to the size, the video frame images corresponding to the "0" elements in the departure points corresponding to several matching indexes ranked in the front are selected as candidate vehicle loading pictures, which greatly improves the recognition accuracy of the vehicle loading pictures. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0039] Figure 1 It is a flowchart of the implementation steps of the vehicle loading picture extraction method provided by an embodiment of the present invention;

[0040] Figure 2 It is a schematic structural diagram of the vehicle loading picture extraction system provided by an embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram of calculating the similarity between the matching sequence and the departure mode sequence. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The technical solutions of the present invention will be further described below with reference to the drawings and through specific embodiments.

[0043] Among them, the drawings are only used for exemplary illustration, showing only schematic diagrams, rather than physical diagrams, and cannot be understood as a limitation of this patent; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, and do not represent the dimensions of the actual products; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0044] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if terms such as "upper", "lower", "left", "right", "inner", "outer", etc. are used to indicate the orientation or positional relationship, it is based on the orientation or positional relationship shown in the accompanying drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and cannot be construed as a limitation of this patent. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0045] In the description of the present invention, unless otherwise clearly specified and limited, if terms such as "connection" are used to indicate the connection relationship between components, this term should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] Express transportation generally uses van trucks, and the package weight is relatively light. Therefore, the upper limit of vehicle loading is usually determined by volume. So, the loading rate described in the present invention refers to the volume loading rate, that is, the volume of the goods in the carriage divided by the total volume of the carriage. It is assumed here that the goods in the carriage are densely packed.

[0047] During the process of sorting and loading goods, the vehicle needs to stop at the loading bay. The loading workers use the assembly line to load the goods into the carriage. There is a camera above each bay, which can record the video of the bay in real time.

[0048] The loading time of the vehicle is relatively long, generally more than 1 hour. After the vehicle finishes loading, the compartment door is usually closed first and then the vehicle drives away from the loading bay, or it may drive away from the bay without closing the compartment door. After the vehicle finishes loading, the picture before closing the compartment door (this picture is simply referred to as the "vehicle loading picture") is used for loading rate identification to ensure the accuracy of loading rate identification. The existing method for automatically extracting vehicle loading pictures is as follows: When the vehicle drives into the loading bay, timing starts. After the set loading duration is reached, the loading picture of the vehicle is taken as the vehicle loading picture after loading. For example, if the vehicle drives into the bay at 10:00 and the set loading duration is 1 hour, then at 11:00, the loading picture of the vehicle is taken as the vehicle loading picture after loading. However, if the total loading duration is, for example, 1 hour and 10 minutes, the vehicle loading picture should be the loading picture taken at 11:10. Although the impact of the vehicle loading pictures taken at 11:10 and 11:00 on the loading rate calculation of a single vehicle is not significant, for the loading rate calculation of the transportation tasks of the entire logistics network, the calculation error of the loading rate of a single vehicle will be amplified, affecting the overall loading rate assessment of the logistics network. A more serious situation is that when the set loading time is, for example, 1 hour and the actual loading duration is 50 minutes, if an image is still taken after the set loading duration is reached, since the vehicle has already finished loading and left, it will result in the inability to capture the vehicle loading picture, which has a greater impact on subsequent loading rate identification. Therefore, a more accurate method is needed to accurately extract the picture of the vehicle after loading and with the door not closed from the surveillance video as the vehicle loading picture.

[0049] Therefore, the present invention provides a method for extracting vehicle loading pictures, as Figure 1 shown, including:

[0050] Step S1, collect video frame images from the video of the monitored loading bay at a specified time interval and store them; the camera installed at the loading bay is facing the loading compartment door, which can clearly capture the entire compartment and see the cargo loading situation inside the compartment; since the vehicle loading time is usually long, generally exceeding 1 hour, in order to reduce the storage amount of video frame images, it is only necessary to collect video frame images from the surveillance video of the loading bay at an interval of a specified time (preferably 30 seconds). The purpose of collecting video frame images is to determine whether there is a vehicle in the video frame images. In the present invention, the concept of whether there is a vehicle or no vehicle at the loading bay is different from the conventional concept of having a vehicle or no vehicle. Having a vehicle in the present invention means that the vehicle appears at the loading bay and the door is not closed. Other situations will all be regarded as no vehicle. For example, if the vehicle appears at the loading bay but the door is in a closed state, it is also regarded as no vehicle at the bay.

[0051] In the present invention, in order to distinguish the source of the surveillance video, there is a corresponding relationship between the surveillance video and the loading bay. This corresponding relationship is stored in the surveillance video management system, and this corresponding relationship is represented by Table 1 below:

[0052]

[0053] Table 1

[0054] In step S2, perform binary classification recognition of whether there is a vehicle for each collected video frame image, and obtain a sequence of picture states arranged in the order of image acquisition time related to the loading bay. The elements in the sequence of picture states are represented by "0" and "1". The vehicle state where the vehicle appears at the loading bay and the door is not closed is represented by "0" (i.e., it is judged that there is a vehicle), otherwise it is represented by "1" (i.e., it is judged that there is no vehicle).

[0055] We perform binary classification recognition of whether there is a vehicle for the collected video frame images by training a binary classification model. In this embodiment, the binary classification model uses a common convolutional neural network. However, instead of directly building a convolutional neural network from scratch, we adopt the method of transfer learning to fine-tune the structure and parameters of the model used for other picture classification tasks so that it can be used for the binary classification recognition of whether there is a vehicle in the present invention. This method saves a large amount of model training time. The basic model we adopt is the existing Xception model, with a fully connected layer of 512 neurons (the activation function is relu) added to its top layer, and an output layer with a single neuron (the activation function is sigmoid) added.

[0056] To train the binary classification model, we intercepted 200,000 pictures from all surveillance videos. To cover all possible situations as much as possible, these pictures include various shooting conditions such as day, night, and backlight. In addition, since the position of the camera installed at the loading bay is fixed, when there is no vehicle at the bay, the pictures taken by the same camera are relatively single. Therefore, according to the correlation coefficient between two adjacent pictures, we selected 120,000 pictures with the smallest correlation coefficient (the smaller the correlation coefficient, the greater the difference between two adjacent pictures, and the richer the picture features as the model training samples, which is beneficial to improving the recognition accuracy of the binary classification model). Then, we manually divided these 120,000 pictures into two categories: with vehicle and without vehicle, and used them as the final model training data set. 100,000 of them are used for model training, and the remaining 20,000 are used for model verification. The recognition accuracy of the finally trained binary classification model on the training set and the verification set is above 98%.

[0057] To associate the state recognition result of the video frame image with the corresponding video frame image and the corresponding loading bay, we record the corresponding relationship between the image state recognition result and the video frame image, and between the video frame image and the loading bay through Table 2 below:

[0058]

[0059] Table 2

[0060] Step S3, identify the 0-1 switching points in the picture status sequence as the departure points, and extract m (preferably 59) elements before the departure points and n (preferably 4) elements after the departure points, and then form a matching sequence with the departure points in the order of image acquisition time of the elements.

[0061] For the surveillance videos of each loading bay in a day, there may be multiple 0-1 switching points in the picture status sequence, and these 0-1 switching points may be the departure points. Generally speaking, there is at least a 30-minute loading process for one departure, and there is at least 2.5 minutes until the arrival of the next vehicle. Therefore, in an ideal state, assuming that the binary classification model correctly identifies the picture status of all video frame images collected from the same surveillance video, then the picture status sequence corresponding to a vehicle should at least contain 60 "0"s and 5 "1"s (assuming one frame is collected every 30 seconds, the 30-minute loading process should at least contain 60 "0"s, and the 2.5-minute waiting process should at least contain 5 "1"s). The last picture with a vehicle (the last "0" among the 60 "0"s) is the vehicle loading picture we need to extract. However, in actual situations, the binary classification model may make classification errors. For example, there should be 59 "0"s before the 0-1 switching point, but due to misidentification by the binary classification model, there may be only 56 "0"s before the 0-1 switching point, and the other 3 are "1"s. Or there should be at least 4 "1"s after the 0-1 switching point, but due to misclassification by the binary classification model, there are only 3 "1"s after the 0-1 switching point, both of which cannot meet the ideal conditions of 59 "0"s before the 0-1 switching point and 4 "1"s after it.

[0062] Therefore, to address the problem that the extraction accuracy of vehicle loading pictures may be low due to classification errors as described above, the vehicle loading picture extraction method provided in this embodiment further includes:

[0063] Step S4, align the matching sequence with a departure mode sequence of the same length and perform similarity matching. After arranging the calculated matching indices in ascending order, select the video frame images corresponding to the "0" elements at the departure points corresponding to several matching indices ranked at the front as candidate vehicle loading pictures.

[0064] The length of the departure mode sequence is preferably 65, and the length of the matching sequence is also 65. The method of aligning the matching sequence with the departure mode sequence is as Figure 3 shown:

[0065] That is, after aligning the departure points (0-1 switching points) in the matching sequence with the matching points (0-1 matching points) in the departure mode sequence, ensure that there is a one-to-one correspondence between the other elements in the matching sequence and the departure mode sequence.

[0066] Perform a similarity match between the matching sequence and the departure pattern sequence, that is: calculate the absolute value of the difference in element values between the matching sequence and the departure pattern sequence. For example, assume that the last 5 elements in the matching sequence are all "1", 58 out of the first 60 are "0", and 2 are "1", while the departure pattern sequence fixedly has 5 "1"s and 60 "0"s. Then, after calculating the absolute value of the difference between the two sequences, the matching index is "2". In the ideal situation, that is, when the binary classification model correctly recognizes the picture states of all video frame images in the matching sequence, there are also 5 "1"s and 60 "0"s in the matching sequence. Then, after calculating the absolute value of the difference between the two sequences, the matching index is "0". Therefore, the smaller the matching index, the higher the similarity between the two sequences, and the more accurate the vehicle loading pictures extracted from the matching sequence. After sorting the calculated matching indices in ascending order from small to large, select the video frame images corresponding to the "0" elements at the departure points corresponding to several of the top-ranked matching indices as candidate vehicle loading pictures.

[0067] The process of performing the similarity match can be expressed by the following formula (1):

[0068]

[0069] In formula (1), x i represents the value of the i-th element in the matching sequence;

[0070] y i represents the value of the i-th element in the departure pattern sequence;

[0071] f represents the matching index to be solved.

[0072] After arranging the matching indices of the present invention from small to large, select the video frame images corresponding to the "0" elements at several departure points with the top-ranked matching indices as vehicle loading pictures (preferably select p + 3 pictures, where p represents the number of transportation tasks associated with the same loading bay).

[0073] In the present invention, one vehicle executes one transportation task. During a day, there may be multiple vehicles arriving at the same loading bay for loading. Therefore, it is necessary to identify the vehicle loading pictures of each vehicle at the loading bay to determine the vehicle loading pictures associated with each transportation task, which is convenient for subsequent calculation of the loading rate of each transportation task. Therefore, the number of candidate vehicle loading pictures extracted after performing the similarity match on the picture state sequence should be greater than the number of transportation tasks associated with this loading bay. In the present invention, the number of candidate vehicle loading pictures extracted after performing the similarity match on the picture state sequence satisfies the constraint condition expressed by the following formula (2):

[0074] c ≤ p + 3 Formula (2)

[0075] In formula (2), c represents the number of vehicle loading pictures extracted as candidates after similarity matching of the picture status sequence;

[0076] p represents the number of transportation tasks associated with the same loading bayonet.

[0077] One vehicle loading picture is associated with one vehicle, and one vehicle is associated with one transportation task. Therefore, one vehicle loading picture should be associated with one transportation task. However, the number of candidate vehicle loading pictures is not necessarily the same as the number of transportation tasks associated with the same loading bayonet. So, it is necessary to eliminate the wrongly extracted candidate pictures. The elimination method is as follows:

[0078] When the acquisition time interval between the first vehicle loading picture and the second vehicle loading picture extracted as candidates is less than the preset time interval (the time interval is preferably 30 minutes because the time interval between the end of loading for two consecutive transportation tasks should be at least 30 minutes), then keep the first vehicle loading picture acquired earlier and eliminate the second vehicle loading picture acquired later. Keeping the first vehicle loading picture acquired earlier and eliminating the second vehicle loading picture acquired later is because the probability that the first vehicle loading picture acquired earlier is a real vehicle loading picture is greater than the probability that the second vehicle loading picture acquired later is real.

[0079] After the candidate vehicle loading pictures are finally determined, we need to associate the vehicle loading pictures with the corresponding transportation tasks. For this purpose, as Figure 1 shown, the vehicle loading picture extraction method provided by the present invention further includes:

[0080] Step S5, associate the vehicle loading pictures with the corresponding transportation tasks according to the association time between the transportation tasks and the loading bayonets. Table 3 below is the association information between the loading bayonets and the transportation tasks.

[0081]

[0082] Table 3

[0083] The method for the present invention to associate the vehicle loading pictures with the transportation tasks is: select the vehicle loading picture closest to the association time between the transportation task and the loading bayonet as the vehicle loading picture having an association relationship with the transportation task. For example, if the association time between the transportation task and the loading bayonet is 16:00:00 (the association time between the transportation task and the loading bayonet is generally set to when the vehicle loading is almost finished), and if the acquisition time of one candidate vehicle loading picture is 16:05:00 and the acquisition time of another vehicle loading picture is 17:10:00, then select the picture at 16:05:00 as the vehicle loading picture having an association relationship with the transportation task.

[0084] In summary, the present invention converts the video frame sequence collected from the video of the monitoring loading bay into a picture state sequence, identifies the departure point in the picture state sequence, then forms a matching sequence containing the departure point, makes a similarity match between the matching sequence and the departure mode sequence, and sorts the calculated matching indices in ascending order from small to large. Then, the video frame images corresponding to the "0" elements among the departure points corresponding to several of the sorted matching indices are selected as candidate vehicle loading pictures, greatly improving the recognition accuracy of vehicle loading pictures.

[0085] The present invention also provides a vehicle loading picture extraction system, which can implement the above vehicle loading picture extraction method. As Figure 2 shown, the system includes:

[0086] An image acquisition module, configured to acquire video frame images from the video of the monitoring loading bay at a specified time interval and store them in a picture database;

[0087] A vehicle state recognition module, connected to the image acquisition module, configured to perform binary classification recognition of whether there is a vehicle on each acquired video frame image, and obtain a picture state sequence arranged in the order of image acquisition time related to the loading bay. The elements in the picture state sequence are represented by "0" and "1". The vehicle state where the vehicle appears at the loading bay and the vehicle door is not closed is represented by "0", otherwise it is represented by "1";

[0088] A departure point recognition module, connected to the vehicle state recognition module, configured to identify the 0-1 switching point in the picture state sequence as the departure point;

[0089] A matching sequence generation module, respectively connected to the departure point recognition module and the vehicle state recognition module, configured to extract m elements before the departure point and n elements after it, and then form a matching sequence with the elements arranged in the order of image acquisition time with the departure point;

[0090] A similarity matching module, connected to the matching sequence generation module, configured to align the matching sequence with an equal-length departure mode sequence and perform similarity matching to obtain the matching index of the matching sequence and the departure mode sequence;

[0091] A candidate picture extraction module, respectively connected to the image acquisition module and the similarity calculation module, configured to sort the calculated matching indices in ascending order, and select the video frame images corresponding to the "0" elements among the departure points corresponding to several of the sorted matching indices as candidate vehicle loading pictures;

[0092] A picture association module, connected to the candidate picture extraction module, configured to associate the vehicle loading pictures with the corresponding transportation tasks according to the association time between the transportation tasks and the loading bay.

[0093] It should be noted that the above specific embodiments are only the preferred embodiments of the present invention and the technical principles applied. Those skilled in the art should understand that various modifications, equivalent substitutions, changes, etc. can be made to the present invention. However, as long as these transformations do not depart from the spirit of the present invention, they should be within the protection scope of the present invention. In addition, some terms used in the specification and claims of this application are not restrictive, but are only for the convenience of description.

Claims

1. A method for extracting vehicle loading pictures, characterized in that the steps include: S1, collecting and storing video frame images from the video monitoring the loading gate at a specified time interval; S2, performing binary classification recognition of the presence or absence of a vehicle on each of the captured video frame images, and obtaining a picture state sequence associated with the loading bayonet arranged in the order of image capture time, wherein the elements in the picture state sequence are represented by "0" and "1", and the vehicle state when the vehicle appears at the loading bayonet and the door is not closed is represented by "0", and otherwise it is represented by "1"; S3, identifying the 0-1 switching point in the image state sequence as the departure point, and extracting m elements before the departure point and n elements after the departure point to form a matching sequence with the departure point in which the elements are arranged in the order of image acquisition time; S4, aligning the matching sequence with a departure pattern sequence of equal length for similarity matching, and arranging the calculated matching indexes in ascending order, selecting the video frame image corresponding to the "0" element in the departure points corresponding to several matching indexes ranked first as a candidate vehicle loading picture, wherein the departure pattern sequence is a standard feature sequence that characterizes vehicle loading and departure characteristics.

2. The vehicle loading picture extraction method according to claim 1, wherein The vehicle loading picture extraction method also includes the steps of: S5, associating the vehicle loading picture with the corresponding transport task according to the association time between the transport task and the loading checkpoint.

3. The vehicle loading picture extraction method according to claim 1, characterized in that, The specified time interval in step S1 is 30 seconds.

4. The vehicle loading picture extraction method according to claim 1, wherein In step S3, m=59.

5. The vehicle loading picture extraction method according to any one of claims 1-4, characterized in that, In step S3, n=4.

6. The vehicle loading picture extraction method according to claim 1, wherein In step S4, the matching sequence is matched with the departure mode sequence by similarity using the following formula (1): In formula (1), x i represents the value of the o-th element in the said matching sequence; y i represents the value of the i-th element in the departure mode sequence; f represents the matching index to be solved.

7. The vehicle loading picture extraction method according to claim 1, wherein, The number of the vehicle loading pictures extracted as candidates after similarity matching of the picture state sequence satisfies the constraint condition expressed by the following formula (2): c≤p+3 formula (2) In formula (2), c represents the number of the vehicle loading pictures extracted as candidates after similarity matching of the picture state sequence; p represents the number of the transport tasks associated with the same loading bayonet.

8. The vehicle loading picture extraction method according to claim 1 or 7, characterized in that When the collection time interval between the first vehicle loading picture and the second vehicle loading picture extracted as candidates in step S4 is less than the preset time interval, the first vehicle loading picture collected first is retained and the second vehicle loading picture collected later is discarded.

9. The vehicle loading picture extraction method according to claim 8, wherein The preset time interval is 1800s.

10. A vehicle loading picture extraction system, which can implement the vehicle loading picture extraction method described in any one of claims 1-9, characterized in that, The system comprises: An image acquisition module is used to acquire video frame images from the video monitoring the loading gate at a specified time interval and store them in an image database; A vehicle state recognition module is connected to the image acquisition module and is used to perform binary classification recognition of the presence or absence of a vehicle on each of the acquired video frame images, and obtain a picture state sequence associated with the loading bayonet arranged in the order of image acquisition time, wherein the elements in the picture state sequence are represented by "0" and "1", and the vehicle state in which the vehicle appears at the loading bayonet and the door is not closed is represented by "0", otherwise it is represented by "1"; A departure point recognition module, connected to the vehicle state recognition module, for recognizing a 0-1 switching point in the image state sequence as a departure point; The matching sequence generation module is respectively connected to the departure point recognition module and the vehicle status recognition module, and is used to extract m elements before the departure point and n elements after the departure point, and then form a matching sequence with the elements arranged in the order of image acquisition time with the departure point; The similarity matching module is connected to the matching sequence generation module, and is used to align the matching sequence with the departure mode sequence of the same length and then perform similarity matching to obtain the matching index of the matching sequence and the departure mode sequence. The departure mode sequence is a standard feature sequence representing the loading and departure characteristics of the vehicle; The candidate picture extraction module is respectively connected to the image acquisition module and the similarity matching module, and is used to sort the calculated matching indexes in ascending order, and then select the video frame images corresponding to the "0" elements at the departure points corresponding to several matching indexes ranked in the front as the candidate vehicle loading pictures; The picture association module is connected to the candidate picture extraction module, and is used to associate the vehicle loading pictures with the corresponding transportation tasks according to the association time between the transportation tasks and the loading bayonet.

Images