Method for detecting passenger carrying of boxcar

Abstract

According to the freight car manned detection method provided by the invention, all targets such as freight cars and portraits in the to-be-recognized picture are found through target detection, the vehicle license plate of the target vehicle is recognized, and the image center point coordinate of each target is constructed; and solving and recording the distance between the central point of the target portrait and the central point of the target vehicle image in the to-be-recognized picture. Finding out an auxiliary judgment image associated with the to-be-recognized image, detecting targets such as a truck and a portrait for an auxiliary judgment image frame containing a target vehicle, and solving the distance between the target portrait in each frame of image and the central point of the target vehicle; and finally, calculating the distance between the portrait center in each frame and the central point of the truck, and if the distance error does not exceed a set threshold value, judging that someone moves along with the vehicle and the target vehicle has an illegal behavior that the carriage carries people.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic control, and in particular to a method for detecting passengers in a truck compartment. Background Art

[0002] Truck compartments do not have seats, seat belts and other facilities specially designed for passengers, and are not suitable for carrying passengers. Moreover, freight vehicles are generally large in size and have a high center of gravity. Once a traffic accident occurs, passengers are likely to be seriously injured. my country's laws and regulations clearly stipulate that it is prohibited to carry passengers in truck compartments, and those who violate the regulations on carrying passengers will face fines, points deduction and other penalties. However, due to the demand for labor in rural areas, plantations, construction sites and other areas, and the inconvenience of travel, freight trucks are often used as a means of carrying passengers, and illegal carrying of passengers is rampant. At present, the detection of people in truck compartments is mostly through the use of capture equipment to capture pictures and then identify them based on image recognition technology, and issue a warning message after finding the suspected vehicle. However, in actual work, it was found that due to the poor viewing angle of the capture camera, there was interference from pedestrians, electric vehicle drivers and motorcycle drivers outside the vehicle. For example, as shown in the attached figure of the specification Figure 4 As shown in the figure, when the image to be identified is captured by the snapshot camera, the pedestrian riding an electric bike next to the truck is just located in the truck compartment, and the height is similar to the usual sitting position, but the electric bike cannot be seen from the snapshot angle. At this time, the truck in the image to be identified will be mistakenly identified as a state where there are people in the compartment. Therefore, the automatic identification method of truck passengers implemented by existing image recognition technology has a high misidentification rate in actual applications due to the angle of the snapshot camera and external interference factors, resulting in serious business error warnings, and a lot of manual work is required for identification. Summary of the invention

[0003] In order to solve the problem of high recognition error rate due to external interference in the existing problem of identifying occupants in truck compartments based on monitoring equipment, the present invention provides a method for detecting occupants in truck compartments, which does not require additional hardware, effectively solves interference factors such as pedestrians, non-motor vehicle drivers, and motorcycle drivers on urban roads, and greatly improves the recognition accuracy.

[0004] The technical solution of the present invention is as follows: a method for detecting people in a truck compartment, comprising the following steps:

[0005] S1: Obtain the image to be identified and detect all target objects in the image;

[0006] The target objects include: vehicles and human figures, which are denoted as: target vehicles and target human figures;

[0007] It is characterized in that it also includes the following steps:

[0008] S2: Identify the vehicle type corresponding to the target vehicle according to the detected target vehicle;

[0009] If the vehicle type is a freight vehicle, step S3 is implemented;

[0010] Otherwise, if the target vehicle is not a freight vehicle, no subsequent detection is performed on the target vehicle;

[0011] S3: Identify the vehicle license plate number of the target vehicle and query its motor vehicle registration information;

[0012] If the license plate number of the target vehicle cannot be identified, no further detection will be performed;

[0013] If the license plate of the target vehicle can be identified, and the vehicle type in the registration information of the vehicle is the target freight vehicle type, then proceed to S4;

[0014] S4: finding the image center coordinates of the target portrait and the target vehicle according to the contour coordinates of the target portrait and the target vehicle in the image to be identified;

[0015] Determine whether the target portrait coordinates fall within the truck image area of ​​the target vehicle, if so, proceed to S5, otherwise end;

[0016] S5: Calculate the pixel distance between the image center coordinates of the target portrait and the image center coordinates of the target vehicle, which is recorded as: center distance;

[0017] S6: extract auxiliary judgment images of the target image, record the auxiliary judgment images one by one as: to-be-recognized images, and repeat steps S1 to S5;

[0018] Each center distance obtained is calculated and compared with a preset error threshold. If all center distances are less than the error threshold, it is determined that a person is accompanying the vehicle and the target vehicle is suspected of illegally carrying a person.

[0019] The method for obtaining the auxiliary judgment image is:

[0020] If the data source of the image to be identified is a road checkpoint picture, extract the road checkpoint pictures containing the target vehicle within a specified similar time period on the same road to obtain the auxiliary judgment image;

[0021] If the data source of the image to be identified is a road monitoring video, the video frame of the time period in which the image to be identified is located is extracted to obtain the image for auxiliary judgment.

[0022] It is further characterized by:

[0023] In step S2, the target freight vehicle types include: gate type, plate type, flatbed type and dump type;

[0024] In step S4, the image center coordinates P of the target portrait are calculated. 人 and the image center coordinates P of the target vehicle 车 The calculation method is as follows:

[0025]

[0026]

[0027] in, is the coordinate of the upper left corner of the target circumscribed rectangular frame in the image pixel coordinate system of the target portrait, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system;

[0028] is the coordinate of the upper left corner of the target vehicle's bounding rectangle in the image pixel coordinate system, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system;

[0029] is the image center coordinate of the target portrait, is the coordinate of the center point of the truck image;

[0030] In step S4, if all of the following conditions are met, it is determined whether the target portrait coordinates fall within the truck image area of ​​the target vehicle;

[0031] Condition 1:

[0032]

[0033] Condition 2: Set the image center coordinates P of the target portrait 人 Connect the line with the image coordinate origin to get the portrait line;

[0034] Connect the lower left coordinate point and the upper right coordinate point of the rectangular frame of the truck area of ​​the target vehicle to the image coordinate origin respectively to obtain two vehicle connection lines;

[0035] Assume that the angles between the portrait line and the two vehicle lines are α and β respectively;

[0036] Then: both α and β are greater than the preset angle threshold;

[0037] The angle threshold is set to: 10°;

[0038] In condition 2, the calculation method of α and β is:

[0039]

[0040]

[0041] In step S5, the center distance D is calculated as follows:

[0042]

[0043] in, is the coordinate of the center point of the portrait image, is the coordinate of the center point of the truck image;

[0044] In step S6, the method for determining whether the person is moving with the vehicle is:

[0045] a1: Calculate the pixel distance between the center point of the portrait and the center point of the truck image in each image to be identified, denoted as: center distance D i ,

[0046] Where i is the number of images to be identified involved in the calculation;

[0047] a2: Calculate the center distance D i With D i The difference in the mean values ​​DIFF:

[0048]

[0049] a3: Compare all DIFFs i and the preset threshold value TH for judging the presence of a person or a vehicle;

[0050] If all the DIFF i If they are all smaller than TH, it means that the position between the person and the truck is relatively stable, the person moves with the vehicle, and the truck is suspected of carrying people illegally;

[0051] Otherwise, it is determined that the truck is not suspected of carrying passengers illegally.

[0052] The present application provides a method for detecting people in a truck compartment. The method detects all trucks, people and other targets in the image to be identified through target detection, identifies the vehicle license plate of the target vehicle, constructs the coordinates of the center point of the image of each target, and solves and records the distance between the center point of the target person in the image to be identified and the center point of the target vehicle image. Then find the auxiliary judgment image associated with the image to be identified, detect trucks, people and other targets for the auxiliary judgment image frame containing the target vehicle, and solve the distance between the center point of the target person and the target vehicle in each frame. Finally, the distance between the center of the person in each frame and the center point of the truck is calculated. If the distance error does not exceed the set threshold, it is determined that someone is accompanying the vehicle movement, and the target vehicle has illegal behavior of carrying people in the compartment. The method for detecting people in a truck compartment provided by the present invention can effectively solve the interference factors such as pedestrians outside the vehicle, non-motor vehicle drivers, and motorcycle drivers on urban roads by using the auxiliary judgment image and the accompanying movement of people and vehicles at the same time, greatly improving the accuracy of recognition and reducing system error business warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the method for detecting passengers in a truck compartment of the present application;

[0054] Figure 2 is a schematic diagram of image coordinates;

[0055] Figure 3 A schematic diagram showing the relationship between the target portrait coordinates and the truck image area of ​​the target vehicle;

[0056] Figure 4 This is an example of an external human figure interfering with the passengers in a truck compartment. DETAILED DESCRIPTION

[0057] like Figure 1 As shown, the present application includes a method for detecting whether a truck compartment has passengers, which includes the following steps.

[0058] S1: Obtain the image to be identified, and find all target objects in the image based on the target monitoring algorithm in the existing image recognition technology;

[0059] The target objects include: vehicles and human portraits, which are recorded as: target vehicles and target human portraits.

[0060] In practical applications, the image to be identified can use road checkpoint pictures, or extract video image frames from road surveillance videos.

[0061] S2: Identify the vehicle type corresponding to the target vehicle according to the detected target vehicle;

[0062] If the vehicle type is a freight vehicle, step S3 is implemented;

[0063] Otherwise, if the target vehicle is not a freight vehicle, no subsequent detection is performed on the target vehicle.

[0064] In real life, trucks are classified into the following types according to the style of the compartment: barge type, plate type, flatbed type, dump type, box type, container type and tank type. Among trucks, the types of vehicles that have enough space to carry people, i.e., the target freight vehicle types in this application, include barge type, plate type, flatbed type and dump type.

[0065] Therefore, if the target vehicle is a gate-type, flatbed-type, or dump truck, subsequent judgment will be made; if the target vehicle is a van-type, container-type, or tank truck, subsequent judgment will no longer be made.

[0066] S3: Identify the vehicle license plate number of the target vehicle and query its motor vehicle registration information;

[0067] If the license plate number of the target vehicle cannot be identified, no further detection will be performed;

[0068] If the license plate of the target vehicle can be identified, and the vehicle type in the registration information of the vehicle is found to belong to the target freight vehicle type, that is, it belongs to: gate type, plate type, flatbed type and dump type, then S4 is performed.

[0069] Vehicles whose license plate numbers cannot be identified will be processed in other vehicle management procedures and will not be monitored by this application.

[0070] The specific target vehicle type and license plate recognition method is implemented based on the algorithm in the prior art, such as building a recognition model through a binary classification model or a neural network model in a machine learning model, or recognizing license plates and license plate numbers based on traditional image recognition algorithms.

[0071] S4: finding the image center coordinates of the target portrait and the target vehicle according to the contour coordinates of the target portrait and the target vehicle in the image to be identified;

[0072] Determine whether the target portrait coordinates fall within the truck image area of ​​the target vehicle, if so, proceed to S5, otherwise, end. Figure 2 As shown, in the xy coordinate system, for the identified target vehicle and target portrait, an external rectangular frame is drawn according to the maximum and minimum values ​​in the contour coordinates.

[0073] In step S4, the image center coordinates P of the target portrait are calculated. 人 and the image center coordinates P of the target vehicle 车 The calculation method is as follows:

[0074]

[0075]

[0076] in, is the coordinate of the upper left corner of the target circumscribed rectangular frame in the image pixel coordinate system of the target portrait, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system;

[0077] is the coordinate of the upper left corner of the target vehicle's bounding rectangle in the image pixel coordinate system, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system;

[0078] is the image center coordinate of the target portrait, are the coordinates of the center point of the truck image.

[0079] In step S4, if all of the following conditions are met, it is determined whether the target portrait coordinates fall within the truck image area of ​​the target vehicle;

[0080] Condition 1:

[0081]

[0082] Condition 2: Set the image center coordinates P of the target portrait 人 Connect the line with the image coordinate origin to get the portrait line;

[0083] Connect the lower left coordinate point and the upper right coordinate point of the rectangular frame of the truck area of ​​the target vehicle to the image coordinate origin respectively to obtain two vehicle connection lines;

[0084] Assume that the angles between the portrait line and the two vehicle lines are α and β respectively;

[0085] Then: both α and β are greater than the preset angle threshold.

[0086] like Figure 3 As shown, the lower left coordinate point and the upper right coordinate point of the rectangular frame of the truck area of ​​the target vehicle are point A and point B respectively, and the image center of the target portrait is point C. Then, the image coordinate origin O is connected with the three points respectively to obtain three lines OA, OB and OC. The angle between OA and OC is α, and the angle between OC and OB is β.

[0087] In real life, when a truck compartment is carrying people, except that the people will be completely within the area of ​​the truck, the relative position between the people and the truck will basically not change, and the people will be located in the compartment, not in a special position such as the front or rear of the vehicle. Therefore, in this application, in addition to controlling the target portrait to be completely inside the target vehicle through condition 1, an angle threshold is set through condition 2 to limit the relative position of the person and the front and rear of the vehicle. In this way, some passers-by who interfere with the outside world can be excluded. The angle threshold is adaptively set according to the actual length of the vehicle. In this embodiment, the angle threshold is set to 10 degrees according to the length of existing common trucks. For example: the α calculated on a certain image to be identified is less than 10 degrees, indicating that the target portrait detected in the image is too close to the front position, indicating that the probability that it is an unrelated passer-by is very high.

[0088] In condition 2, the calculation method of α and β is:

[0089]

[0090]

[0091] S5: Calculate the pixel distance between the image center coordinates of the target portrait and the image center coordinates of the target vehicle, which is recorded as: center distance.

[0092] In step S5, the center distance D is calculated as follows:

[0093]

[0094] in, is the coordinate of the center point of the portrait image, are the coordinates of the center point of the truck image.

[0095] S6: extract auxiliary judgment images of the target image, record the auxiliary judgment images one by one as: to-be-recognized images, and repeat steps S1 to S5;

[0096] Each center distance obtained is calculated and compared with a preset error threshold. If all the center distances are smaller than the error threshold, it is determined that a person is accompanying the vehicle and the target vehicle is suspected of illegally carrying people.

[0097] The method for obtaining the auxiliary judgment image is as follows:

[0098] If the data source of the image to be identified is a road checkpoint picture, extract the road checkpoint pictures containing the target vehicle within a specified similar time period on the same road to obtain the auxiliary judgment image;

[0099] If the data source of the image to be identified is a road monitoring video, the video frame of the time period in which the image to be identified is located is extracted to obtain the image for auxiliary judgment.

[0100] In specific applications, if the capture device is a road checkpoint device that records images, all checkpoint devices on the same road code within 1 hour are selected according to the checkpoint device registration information, and the checkpoint image of the truck with the license plate number identified in the first frame is extracted, and the distance between the portrait and the center point of the corresponding truck image is calculated and recorded; if the capture device is a video device, the video image frames are extracted at intervals of 1 second, and the distance between the portrait and the center point of the corresponding truck image is calculated and recorded.

[0101] In step S6, the method for determining whether the person is moving with the vehicle is:

[0102] a1: Calculate the pixel distance between the center point of the portrait and the center point of the truck image in each image to be identified, denoted as: center distance D i ,

[0103] Where i is the number of images to be identified involved in the calculation;

[0104] a2: Calculate the center distance D i With D i The difference in the mean value DIFF i :

[0105]

[0106] a3: Compare all DIFFs i and the preset threshold value TH for judging the presence of a person or a vehicle;

[0107] If all the DIFF i If they are all smaller than TH, it means that the position between the person and the truck is relatively stable, the person moves with the vehicle, and the truck is suspected of carrying people illegally;

[0108] Otherwise, it is determined that the truck is not suspected of carrying passengers illegally.

[0109] After eliminating pedestrians at the front and rear of the vehicle through the above conditions 1 and 2, if Figure 4The pedestrians shown are completely located in the car compartment and cannot be excluded. Therefore, in this application, the method of judging whether a person moves with a vehicle is used to further exclude external interference portraits. Generally speaking, even if the position of the external interference portrait at a certain point in time is consistent with the phenomenon of a truck compartment carrying people, when the truck compartment is actually carrying people, the relative positions of the person and the car are fixed. The positions of the externally interfering pedestrians and vehicles will change. Therefore, in this application, the center point of the target human portrait and the center point of the truck image will be calculated first, and then the auxiliary judgment image including the same target vehicle will be used to jointly judge whether the relative position of the person and the car has changed within a period of time. If there is no change within a period of time, it can be determined that the target vehicle is suspected of carrying people in the compartment.

[0110] However, in actual applications, due to the differences in the hardware performance of the monitoring equipment used to collect images, when capturing images during vehicle operation, the size of the captured images may change as the vehicle moves closer or farther away. The target vehicle and target portrait in the captured image may also be slightly deformed, or the center position of the target portrait may change due to changes in posture. These will cause the pixel distance D between the center point of the portrait and the center point of the truck image to change. i There are slight changes. In order to solve this practical problem, in this application, a threshold value TH is set to control these slight differences by setting the human-vehicle accompanying judgment. The specific TH value is adaptively set according to the installation location and performance of the acquisition equipment. For example, the acquisition equipment installed at the intersection can set the TH value to a smaller value because the vehicles at the intersection usually have a lower speed. The monitoring equipment installed on the highway can adjust the TH value to a larger value because the vehicles on the road have a higher speed. In this embodiment, the TH value corresponding to the monitoring equipment installed on the urban road is set to 50 pixels.

[0111] After using the technical solution of the present invention, the external interfering human images located in the non-carriage area are excluded by conditions 1 and 2, and the external interfering human images located in the car area are excluded by using continuous images through the judgment method of human accompanying vehicle movement. In the judgment method of human accompanying vehicle movement, the threshold TH for judging human-vehicle accompanying movement is set to sum the difference DIFF i Eliminating the interference elements caused by the collection equipment effectively improves the accuracy of behavior recognition of people in freight vehicle compartments.

Claims

1. A method for detecting passengers in a truck compartment, comprising the following steps: S1: Obtain the image to be identified and detect all target objects in the image; The target objects include: vehicles and human figures, which are denoted as: target vehicles and target human figures; It is characterized in that it also includes the following steps: S2: Identify the vehicle type corresponding to the target vehicle according to the detected target vehicle; If the vehicle type is a freight vehicle, step S3 is implemented; Otherwise, if the target vehicle is not a freight vehicle, no subsequent detection is performed on the target vehicle; S3: Identify the vehicle license plate number of the target vehicle and query its motor vehicle registration information; If the license plate number of the target vehicle cannot be identified, no further detection will be performed; If the license plate of the target vehicle can be identified, and the vehicle type in the registration information of the vehicle is the target freight vehicle type, then proceed to S4; S4: finding the image center coordinates of the target portrait and the target vehicle according to the contour coordinates of the target portrait and the target vehicle in the image to be identified; Determine whether the target portrait coordinates fall within the truck image area of ​​the target vehicle, if so, proceed to S5, otherwise end; S5: Calculate the pixel distance between the image center coordinates of the target portrait and the image center coordinates of the target vehicle, which is recorded as: center distance; S6: extract auxiliary judgment images of the target image, record the auxiliary judgment images one by one as: to-be-recognized images, and repeat steps S1 to S5; Each center distance obtained is calculated and compared with a preset error threshold. If all center distances are less than the error threshold, it is determined that a person is accompanying the vehicle and the target vehicle is suspected of illegally carrying a person. The method for obtaining the auxiliary judgment image is: If the data source of the image to be identified is a road checkpoint picture, extract the road checkpoint pictures containing the target vehicle within a specified similar time period on the same road to obtain the auxiliary judgment image; If the data source of the image to be identified is a road monitoring video, the video frame of the time period in which the image to be identified is located is extracted to obtain the image for auxiliary judgment.

2. A method for detecting passengers in a truck compartment according to claim 1, characterized in that: In step S2, the target freight vehicle types include: gate type, flatbed type, flatbed type and dump type.

3. The method for detecting passengers in a truck compartment according to claim 1, characterized in that: In step S4, the image center coordinates P of the target portrait are calculated. 人 and the image center coordinates P of the target vehicle 车 The calculation method is as follows: in, is the coordinate of the upper left corner of the target circumscribed rectangular frame in the image pixel coordinate system of the target portrait, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system; is the coordinate of the upper left corner of the target vehicle's bounding rectangle in the image pixel coordinate system, is the coordinate of the lower right corner of the target's bounding rectangle in the image pixel coordinate system; is the image center coordinate of the target portrait, are the coordinates of the center point of the truck image.

4. A method for detecting passengers in a truck compartment according to claim 3, characterized in that: In step S4, if all of the following conditions are met, it is determined whether the target portrait coordinates fall within the truck image area of ​​the target vehicle; Condition 1: Condition 2: Set the image center coordinates P of the target portrait 人 Connect the line with the image coordinate origin to get the portrait line; Connect the lower left coordinate point and the upper right coordinate point of the rectangular frame of the truck area of ​​the target vehicle to the image coordinate origin respectively to obtain two vehicle connection lines; Assume that the angles between the portrait line and the two vehicle lines are α and β respectively; Then: both α and β are greater than the preset angle threshold.

5. The method for detecting passengers in a truck compartment according to claim 1, characterized in that: The angle threshold is set to: 10°.

6. A method for detecting passengers in a truck compartment according to claim 4, characterized in that: In condition 2, the calculation method of α and β is:

7. A method for detecting passengers in a truck compartment according to claim 3, characterized in that: In step S5, the center distance D is calculated as follows: in, is the coordinate of the center point of the portrait image, are the coordinates of the center point of the truck image.

8. A method for detecting passengers in a truck compartment according to claim 7, characterized in that: In step S6, the method for determining whether the person is moving with the vehicle is: a1: Calculate the pixel distance between the center point of the portrait and the center point of the truck image in each image to be identified, denoted as: center distance D i , where i is the number of images to be identified involved in the calculation; a2: Calculate the center distance D i With D i The difference in the mean values ​​DIFF: a3: Compare all DIFFs i and the preset threshold value TH for judging the presence of a person or a vehicle; If all the DIFF i If they are all smaller than TH, it means that the position between the person and the truck is relatively stable, the person moves with the vehicle, and the truck is suspected of carrying people illegally; Otherwise, it is determined that the truck is not suspected of carrying passengers illegally.

Images