A method, device, system and over-limit detection device for collecting over-limit information

Abstract

The present application provides a method, device, system, and over-limit detection device for collecting over-limit information, relating to the field of road vehicle detection technology. The over-limit information collection method provided in the embodiments of the present application can solve the problem in the prior art whereby multiple independent cameras, when separately identifying features of the front, rear, and body of a vehicle, cannot correlate the recognition results. Because the three cameras provided on the over-limit detection device in the present application have overlapping shooting angles, and by accurately tracking and distinguishing vehicles in the video stream, the recognition results of the three cameras can be accurately correlated to the same vehicle, forming correct and complete vehicle data without causing confusion in the retained evidence.

Description

Technical Field

[0001] The present application relates to the technical field of road vehicle detection, and in particular to a method, device, system and overload detection equipment for collecting overload information. Background Art

[0002] With the continuous advancement of the work of canceling provincial border toll stations on expressways across the country and the comprehensive implementation of entrance weighing and testing at expressway entrances, the requirements of mandatory inspection of trucks and prohibition of overweight vehicles from passing on the road have been implemented. The Ministry of Transport has issued the "Expressway Weighing and Testing Business Specifications and Technical Requirements", requiring expressway operating and management units in all provinces, cities and autonomous regions to be responsible for the deployment of non-stop weighing and testing equipment at entrances and exits, to ensure the stable operation of the equipment and the smooth operation of the network, and to gradually realize non-stop weighing and testing in accordance with unified technical requirements.

[0003] According to business specifications and technical requirements, when a truck passes through an overload detection device, it should automatically collect truck weighing data, front and rear license plate recognition data, and capture front, rear, and side photos and more than 5 seconds of passing video data. To meet the above business and data collection requirements, the design solutions recommended by the Ministry of Transport and currently widely used in China are as follows: Figure 1 As shown. Figure 1 The design shown here requires the installation of weighing equipment (scales), front-end cameras, rear-end cameras, side cameras, and a hard disk recorder in each overload detection lane at a toll station. Vehicles entering the detection lanes are inspected for overloads. Illegally overloaded vehicles are forced to leave the highway, allowing legally loaded vehicles to re-enter.

[0004] Figure 1 The existing design shown can theoretically and ideally be implemented and meet relevant requirements according to timing and queue logic. However, in actual applications, it is difficult to achieve automatic detection without stopping the vehicle due to the influence of many factors such as the geographical environment, equipment installation location, and equipment working status.

[0005] The main deficiencies and defects are as follows:

[0006] (1) The equipment investment is large, the construction is complex, and the engineering workload is large. In addition to the necessary weighing and detection equipment, each inspection lane needs to be equipped with a front camera, a rear camera, a side camera, and a hard disk recording device, which requires a large investment and occupies limited network IP resources. At the same time, each inspection lane needs to be equipped with two L-shaped cantilever columns in the front and back to install the cameras. The geographical environment of each toll station inspection lane is different, and the appropriate installation position of the column needs to be surveyed according to the actual on-site environment. The construction workload is large and the construction is complex.

[0007] (2) Weighing data, vehicle front data, vehicle rear data, and side capture data are generated by independent devices and have no connection. In order to associate and match the detection data of the same vehicle, it can only be achieved by relying on timing and queue logic. Therefore, in actual applications, if the clock of a device is not synchronized, data is not detected, or errors and abnormalities occur in the work, it will cause confusion in the subsequent data queues, and data matching will be very difficult without human intervention. Summary of the Invention

[0008] The purpose of the embodiments of the present application is to provide a method, device, system and over-limit detection equipment for collecting over-limit information. The over-limit detection equipment has a small construction working area and is easy to install. By tracking the vehicle, the recognition results of multiple cameras can be accurately associated with the same vehicle to form correct and complete vehicle data.

[0009] In a first aspect, an embodiment of the present application provides a method for collecting overload information, the method comprising: obtaining a first video stream shot by a front camera, a second video stream shot by a side camera, and a third video stream shot by a rear camera provided on an overload detection device; the overload detection device is provided on one side of a detection lane, the front camera, the side camera, and the rear camera are sequentially installed on the overload detection device according to the direction of vehicle travel, and there is overlap between the shooting angles of the front camera and the side camera, and between the shooting angles of the side camera and the rear camera; after detecting that a vehicle to be detected enters the first video stream, the control The front camera captures a front image of the vehicle to be detected and identifies the front license plate information of the vehicle to be detected; the vehicle target is tracked for the vehicle to be detected in the second video stream, and a vehicle body image of the vehicle to be detected is obtained based on the second video stream; after detecting that the vehicle to be detected in the second video stream has traveled to a preset area in the video screen, the rear camera is controlled to capture a rear image of the vehicle to be detected and identify the rear license plate information of the vehicle to be detected; the vehicle front image, vehicle body image, vehicle rear image, front license plate information and rear license plate information corresponding to the vehicle to be detected are uploaded to the business system.

[0010] In the above scheme, since the three cameras set on the overload detection equipment are installed in sequence according to the direction of vehicle travel and their shooting angles overlap, in the process of analyzing the video stream taken by the camera, by accurately tracking and distinguishing the vehicles in the video stream, the recognition results of the three cameras can be accurately associated with the same vehicle, forming correct and complete vehicle data, which will not cause confusion in the retained evidence.

[0011] In one possible embodiment, the side camera is a fisheye camera, and obtaining the vehicle body image of the vehicle to be detected based on the second video stream includes: extracting multiple frames of vehicle body side images from the second video stream, and stitching the multiple frames of vehicle body side images into one image through image stitching to obtain the vehicle body image.

[0012] The above solution uses a fisheye camera and image stitching technology to perfectly synthesize a vehicle side profile photo, solving the problem in the existing technology that due to the limitations of the side camera's viewing angle and capture range, it is impossible to fully capture the vehicle side profile photo for extra-long vehicles.

[0013] In one possible embodiment, after obtaining the first video stream captured by the front camera of the vehicle set on the over-limit detection equipment, the method further includes: using a target detection model to perform target detection on the first video stream to obtain one or more detection target information, wherein the detection target information includes the category and confidence of the detection target; judging whether there is a detection target in the one or more detection target information whose category is a preset vehicle category and the corresponding confidence is greater than a first preset threshold, and if so, determining that a vehicle to be detected has entered the first video stream.

[0014] In one possible embodiment, the method further includes: performing target detection on the vehicle body image using a target detection model to obtain one or more detection target information, wherein the detection target information includes the category and confidence of the detection target; determining, from the one or more detection target information, a detection target whose category is a tire and whose confidence is greater than a second preset threshold, and determining the number of detection targets that meet the above requirements, and determining the number as the number of axles of the vehicle to be detected.

[0015] In the above scheme, the number of vehicle axles can be accurately identified based on the vehicle body picture, which is conducive to more accurate vehicle detection and charging.

[0016] In a possible implementation, the method further includes: intercepting a vehicle entry video of a preset length from the first video stream, the second video stream, or the third video stream, and uploading the vehicle entry video together with a vehicle front image, a vehicle body image, and a vehicle rear image to the business system.

[0017] In a possible implementation, the method further includes: obtaining weighing data of the vehicle to be inspected from a weighing instrument, and uploading the weighing data together with a front image of the vehicle, a body image of the vehicle, and a rear image of the vehicle to the business system.

[0018] In the second aspect, an embodiment of the present application provides an over-limit detection device, comprising: a pillar; an over-limit detection device arranged on the pillar; the over-limit detection device comprises a front camera, a side camera, a rear camera, a controller and a switch, the front camera, the side camera and the rear camera are respectively connected to the controller, and the controller is connected to the switch; the front camera, the side camera and the rear camera are installed on the side of the pillar in sequence according to a preset direction, and there is an overlap between the shooting angles of the front camera and the side camera, and between the shooting angles of the side camera and the rear camera.

[0019] The above solution integrates multiple cameras into the same overload detection equipment. The overload detection equipment can be installed next to the lane of the inspection lane and can shoot vehicles traveling in the center of the road from the side of the road, so that following vehicles will not be missed; moreover, during construction, only a small area of ​​cement foundation is needed for installation, the construction working area is small, the equipment installation is simple, and construction is easy.

[0020] In a possible implementation manner, the pillar has multiple side surfaces, and the front camera, the side camera, and the rear camera are respectively installed on one of the multiple side surfaces.

[0021] In a third aspect, an embodiment of the present application provides an over-limit information collection device, the device comprising: an acquisition module for acquiring a first video stream shot by a front camera, a second video stream shot by a side camera and a third video stream shot by a rear camera arranged on an over-limit detection device; the over-limit detection device is arranged on one side of a detection lane, the front camera, the side camera and the rear camera are sequentially installed on the over-limit detection device according to the direction of vehicle travel, and there is overlap between the shooting angles of the front camera and the side camera, and between the shooting angles of the side camera and the rear camera; a front vehicle recognition module is used to control the front camera to capture the video stream after detecting that a vehicle to be detected enters the first video stream. Take a front-facing picture of the vehicle to be detected and identify the front license plate information of the vehicle to be detected; a side recognition module is used to track the vehicle to be detected in the second video stream and obtain a body picture of the vehicle to be detected based on the second video stream; a rear-facing recognition module is used to control the rear-facing camera to capture a rear-facing picture of the vehicle to be detected and identify the rear-facing license plate information of the vehicle to be detected after detecting that the vehicle to be detected in the second video stream has traveled to a preset area in the video screen; an information uploading module is used to upload the front-facing picture, body picture, rear-facing picture, front license plate information and rear license plate information corresponding to the vehicle to be detected to the business system.

[0022] In the fourth aspect, an embodiment of the present application provides an over-limit information collection system, comprising: an over-limit detection device and a central control device as described in the third aspect, the over-limit detection device being communicatively connected to the central control device; the over-limit detection device being used to send a first video stream taken by a front camera, a second video stream taken by a side camera, and a third video stream taken by a rear camera arranged on the over-limit detection device to the central control device; the central control device being used to execute the over-limit information collection method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a design scheme for over-limit detection in the existing technology;

[0025] Figure 2 A schematic diagram of an over-limit detection device provided in an embodiment of the present application;

[0026] Figure 3 Another schematic diagram of the over-limit detection device provided in an embodiment of the present application;

[0027] Figure 4 This is a schematic diagram of the layout of the over-limit detection equipment in the embodiment of the present application;

[0028] Figure 5 Flowchart of the method for collecting over-limit information provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of an over-limit information collection device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the central control device provided in an embodiment of the present application.

[0031] Icons: 100-overload detection equipment; 110-column; 120-front camera; 130-side camera; 140-rear camera; 150-controller; 160-switch. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] First embodiment

[0034] like Figure 1 In the existing technical solution shown, the front camera, side camera and rear camera work independently. When the vehicle drives into the capture area set by the camera and the vehicle features can be recognized (for example, the license plate number can be recognized), the capture will be triggered. In the case of dense traffic, close following vehicles, or mixed traffic of large and small vehicles, the probability of incorrect recognition and inability to separate vehicles is very high, resulting in the inability to accurately associate the camera recognition results with the same vehicle, that is, it is impossible to determine whether the recognition results of multiple cameras are the same target vehicle. For example, Figure 1 In the detection lane, two L-shaped cantilever columns are arranged, one in front of and one behind the vehicle. The front camera and the side camera are arranged on the column cross bar in front of the vehicle, and their shooting angle is to shoot the vehicle below from above. The rear camera is arranged on the column cross bar behind the vehicle. When a large truck passes by, if there is a small passenger car following closely behind the truck, due to the arrangement position and shooting angle of the front camera and the side camera, the large truck will completely block the small passenger car in the captured image. Since each camera works independently, as long as any camera does not capture the small passenger car, all subsequent captured images will be chaotic, which will cause confusion in the retained evidence.

[0035] Therefore, this embodiment provides an over-limit detection device, such as Figure 2 and Figure 3 As shown, the over-limit detection device 100 includes: a pillar 110 and an over-limit detection device mounted on the pillar 110. The over-limit detection device includes a front camera 120, a side camera 130, a rear camera 140, a controller 150, and a switch 160. The front camera 120, the side camera 130, and the rear camera 140 are respectively connected to the controller 150, which is in turn connected to the switch 160. The front camera 120, the side camera 130, and the rear camera 140 are sequentially mounted on the sides of the pillar 110 in a predetermined orientation. The shooting angles of the front camera 120 and the side camera 130, as well as the shooting angles of the side camera 130 and the rear camera 140, overlap. In the over-limit detection device 100, the front camera 120, the side camera 130, and the rear camera 140 are intelligently integrated. These three cameras are video cameras that can capture images within their fields of view and capture images. In practical applications, the overload detection device 100 is arranged on one side of the detection lane, such as in a safety island next to the inner lane of the detection lane.

[0036] Optionally, the switch 160 is also connected to a central control device, and the front camera 120, the side camera 130, and the rear camera 140 are interconnected with the central control device through the switch 160. Therefore, multiple cameras work together in the same working scenario. The central control device can send control instructions to the designated camera through the network to control the camera's capture, video streaming, etc. The camera's captured images, video streams, and other data are also sent to the central control device through the network. Since the overload detection device 100 is set on one side of the detection lane, the camera's shooting angle is to shoot vehicles traveling in the center of the road from the side of the road, and since there is overlap between the shooting angles of multiple cameras, even in the case of dense traffic, vehicles following closely, or mixed traffic, as long as there is a certain gap between the vehicles, it is possible to accurately capture videos and pictures of the vehicles traveling without omissions.

[0037] Optional, such as Figure 2 As shown, the pillar has multiple side surfaces, and the front camera, side camera and rear camera are respectively installed on one of the multiple side surfaces.

[0038] Figure 4 This is a schematic diagram of the layout of the over-limit detection equipment in this application, from Figure 4 As can be seen, when a vehicle is driving on the detection lane, it first enters the field of view of the front camera, then gradually exits the field of view of the front camera and enters the field of view of the side camera (because the fields of view of the front camera and side camera overlap, the vehicle appears in the video images of both cameras at the same time for a certain period of time), and then gradually exits the field of view of the side camera and enters the field of view of the rear camera (because the fields of view of the side camera and rear camera overlap, the vehicle appears in the video images of both cameras at the same time for a certain period of time). The front camera, side camera, and rear camera capture video streams in real time and transmit the video streams via the network to the central control device, which analyzes and processes the video streams.

[0039] Optionally, the column has four side surfaces, three of which are each equipped with a camera, and a rear door is provided on the side not equipped with a camera. The controller and switch are located in the storage space formed by the rear door and the column. The side of the rear door connected to the column is hingedly connected to the main body of the column. The rear door facilitates installation and maintenance of devices installed in the column.

[0040] Among the three cameras in this embodiment, the side camera used to shoot the side of the vehicle body is a fisheye camera, so that the side camera has a sufficiently large visual range in the vertical direction. For example, a fisheye panoramic camera with a shooting angle of up to 160 degrees can be selected. In this way, for large vehicles traveling on the detection lane, the side camera can capture the upper and lower edges of the large vehicle to obtain a complete picture of the vehicle body. The front camera used to shoot the front of the vehicle and the rear camera used to shoot the rear of the vehicle can be ordinary cameras.

[0041] Optionally, multiple limit holes are provided on the supporting bottom surface of the column. If the existing toll station inspection lane safety island has a concrete floor, the overload detection equipment can be installed by simply using these multiple limit holes and matching screws. Furthermore, even if the floor inside the safety island is not concrete, a cement foundation of a certain size (for example, 320mm*257mm) can be constructed in an appropriate location for installation. This reduces the construction area, simplifies equipment installation, and facilitates construction.

[0042] Optionally, the over-limit detection device further includes a fill light connected to the controller. The number of fill lights can be one or more. In one embodiment, a fill light is provided next to each camera. The separate fill lights ensure clear camera images and good image quality even at night.

[0043] Optionally, the front, side, and rear cameras are fixed to the side of the pillars via adjustable pitch mounts. The pitch angles of the front, side, and rear cameras can be adjusted via the adjustable pitch mounts. In practice, the pitch angles of the mounts can be flexibly adjusted to suit specific scenarios, such as the location of over-limit detection equipment, adjusting the shooting angles of the cameras mounted thereon accordingly.

[0044] Optionally, the column is formed by die-casting aluminum molds. The casting process requires good process precision, and after the casting is completed, the outer surface of the column can have a certain waterproof ability.

[0045] In some embodiments, the above-mentioned over-limit detection equipment can be combined with the weighing detection equipment to jointly complete the over-limit detection of vehicles; in other embodiments, the over-limit detection equipment can work independently and can be used alone for vehicle feature analysis and data collection to provide data support for vehicle feature library big data analysis, such as using image feature analysis to conduct inspections of vehicles with reversed license plates and vehicles with abnormal vehicle models.

[0046] Second embodiment

[0047] This embodiment provides a method for collecting over-limit information. Before introducing this method, the over-limit information collection system that implements this method will be described. The over-limit information collection system includes: an over-limit detection device and a central control device, wherein the over-limit detection device is provided with a front camera, a side camera, and a rear camera. The front camera, the side camera, and the rear camera are installed on the side of the over-limit detection device in sequence according to the direction of vehicle travel. There is overlap between the shooting angles of the front camera and the side camera, and between the shooting angles of the side camera and the rear camera. In one embodiment, the over-limit detection device can be set on one side of the detection lane, such as in a safety island next to the inner lane of the detection lane.

[0048] The over-limit detection device is communicatively connected to the central control device, and the front camera, side camera, and rear camera are interconnected with the central control device network. The central control device can send control instructions to the designated camera via the network to control the camera's capture, video stream transmission, etc. The camera's captured images, video streams, and other data are also sent to the central control device via the network. The over-limit detection device is used to send a first video stream captured by the front camera, a second video stream captured by the side camera, and a third video stream captured by the rear camera, which are installed on the over-limit detection device, to the central control device; the central control device is used to analyze and process the multiple video streams sent by the over-limit detection device, and to trigger the camera to take photos. The over-limit detection device in this embodiment can be the over-limit detection device provided in the first embodiment.

[0049] Taking the above-mentioned over-limit information collection system as an example, this embodiment specifically describes the provided over-limit information collection method. Figure 5 The flowchart of the over-limit information collection method is shown in FIG. Figure 5 The flowchart shown is described with the central control device as the main body. The method includes the following steps:

[0050] Step 210: Obtain a first video stream captured by a front camera, a second video stream captured by a side camera, and a third video stream captured by a rear camera provided on the overload detection device.

[0051] The central control device obtains the first video stream, the second video stream and the third video stream from the over-limit detection device in real time.

[0052] Step 220: After detecting that a vehicle to be detected enters the first video stream, the front camera of the vehicle is controlled to capture a front image of the vehicle to be detected, and identify the front license plate information of the vehicle to be detected.

[0053] After acquiring multiple video streams in step 210, the central control device analyzes the first video stream from the vehicle's front camera in real time and performs target detection on the first video stream. Optionally, the step of performing target detection on the first video stream includes: first, performing target detection on the first video stream using a target detection model to obtain one or more detection target information, each detection target information including the category and confidence level of the detection target; then, determining whether any of the one or more detection target information contains a detection target of a preset vehicle category and a corresponding confidence level greater than a first preset threshold; if so, determining that a vehicle to be detected has entered the first video stream.

[0054] In a specific embodiment, the target detection model can be generated according to a preset neural network based on deep learning, for example, an SSD (Single Shot MultiBox Detector) network can be used. The target detection model can be used to detect whether there is a vehicle to be detected entering the video stream. Specifically, the first video stream captured by the front camera is input into the target detection model to obtain one or more detection target information. Each detection target information includes: the category and confidence of the detection target, wherein the category of the detection target may include categories such as passenger cars, trucks, tires and backgrounds; the detection target information also includes: the location of the detection target in the video picture. Based on the obtained location information, real-time tracking of the vehicle to be detected in the first video stream can be achieved.

[0055] Before actual application, the preset neural network is trained in advance using a number of labeled image data, and the training process is stopped when the training end conditions are met to obtain the target detection model.

[0056] After obtaining one or more detection target information, determine whether there are detection targets classified as buses or trucks in these detection target information, and the confidence level is greater than a first preset threshold. If not, determine that no vehicle to be detected enters the first video stream. If yes, determine that a bus or truck enters the first video stream.

[0057] Optionally, after determining that a vehicle to be detected enters the first video stream, a detection target information with a preset vehicle category and the highest confidence level is determined from one or more detection target information, and the detection target information is used as the information of the vehicle to be detected appearing in the first video stream (including the category and location of the vehicle).

[0058] After detecting the entry of a vehicle to be detected in the first video stream through the above-described embodiment, the central control device sends a control instruction to the vehicle's front camera via the network, instructing the camera to capture a front-facing image of the vehicle to be detected. After capturing the image, the camera transmits the captured front-facing image of the vehicle to the central control device via the network. Simultaneously, upon detecting the entry of the vehicle to be detected, the central control device automatically identifies the vehicle's front-facing license plate information. This identification of the vehicle's front-facing license plate information can utilize an existing license plate recognition algorithm, which can automatically identify the vehicle's front-facing license plate number based on the first video stream or the captured front-facing image of the vehicle.

[0059] After step 220 , step 230 is continued: vehicle target tracking is performed on the vehicle to be detected in the second video stream, and a vehicle body image of the vehicle to be detected is obtained based on the second video stream.

[0060] After detecting that a vehicle to be detected enters the first video stream, the central control device analyzes the second video stream taken by the side camera in real time, and tracks the vehicle to be detected in the second video stream in real time to determine the position of the vehicle to be detected in the video screen, and at the same time obtains a vehicle body image of the vehicle to be detected based on the second video stream.

[0061] Optionally, the side camera is a fisheye camera, such as a panoramic fisheye camera with a 160-degree field of view. The central control device extracts multiple frames of side images of the vehicle to be inspected from the second video stream and stitches them together into a single image to obtain a vehicle body image. For large, long, and tall vehicles, photographing the vehicle with a conventional camera may result in an incomplete image. However, using a fisheye camera and stitching together multiple frames of side images in the video stream into a complete image ensures that the length, height, and axle information of the vehicle to be inspected are complete.

[0062] This embodiment uses a fisheye camera and image stitching technology to perfectly synthesize a side profile photo of the vehicle. Compared with the existing solution of capturing vehicle photos by a side camera set on a crossbar, it solves the problem of being unable to fully capture the side profile photo of an extra-long vehicle due to the limitations of the side camera's viewing angle and capturing range.

[0063] Step 240: After detecting that the vehicle to be detected in the second video stream has traveled to a preset area in the video screen, the rear camera is controlled to capture a rear image of the vehicle to be detected, and the rear license plate information of the vehicle to be detected is identified.

[0064] The central control device tracks the position of the vehicle to be detected in the second video stream in real time. When the vehicle to be detected reaches a preset area in the video image, the vehicle to be detected is considered to be in an exit state, and subsequent analysis can then be performed. In a specific embodiment, the central control device uses a target detection algorithm to track the second video stream and generates a target frame in real time. The area within the target frame represents the position of the vehicle to be detected in the video image. When the target frame corresponding to the vehicle to be detected intersects with the preset area in the video image or the target frame is completely contained in the preset area, the vehicle to be detected can be considered to have reached the preset area in the video image.

[0065] After detecting that the vehicle to be inspected is in the exit state, the central control device sends a control instruction to the rear camera via the network, instructing the rear camera to capture a picture of the rear of the vehicle to be inspected. After the capture is completed, the rear camera sends the obtained rear image of the vehicle to the central control device via the network. Simultaneously, after detecting that the vehicle to be inspected is in the exit state, the central control device automatically identifies the rear license plate information of the vehicle to be inspected. Recognizing the rear license plate information can utilize an existing license plate recognition algorithm, which can automatically identify the license plate number on the rear of the vehicle based on the third video stream captured by the rear camera or the captured rear image of the vehicle.

[0066] In this embodiment, the front image of the vehicle can be used to determine the vehicle's license plate number and make and model, the rear image can be used to determine the trailer's license plate and check the vehicle's cargo load, and the body image can be used to check and determine the truck's axle type. Furthermore, in accordance with relevant business regulations and technical requirements, these three vehicle images are retained as evidence.

[0067] Step 250: Upload the vehicle front image, vehicle body image, vehicle rear image, front license plate information and rear license plate information corresponding to the vehicle to be detected to the business system.

[0068] The central control device integrates the vehicle front image, vehicle body image, vehicle rear image, front license plate information, and rear license plate information obtained in the previous steps and pushes them to the business system through a preset interface. The business system can be deployed on the central control device or on other physical devices that communicate with the central control device. At the same time, the above image data and license plate information can be saved to local storage as needed.

[0069] In the above solution, the central control device uses a target detection model and image tracking technology to perform vehicle feature analysis and image capture in real time. During the analysis process, it can accurately track and distinguish vehicles in the video footage captured by the side camera. In addition, since the target detection model uses a neural network based on deep learning and has an autonomous learning function, the accuracy of vehicle detection will continue to improve as the number of vehicle analyses continues to increase. This embodiment can accurately detect vehicles in toll station scenarios. Compared with existing technologies, it can effectively avoid problems such as recognition errors and vehicle omissions.

[0070] Optionally, before step 250, the method further includes: intercepting a vehicle entry video of a preset length from the first video stream, the second video stream or the third video stream, and uploading the vehicle entry video together with the vehicle front image, the vehicle body image, and the vehicle rear image to the business system in step 250.

[0071] Furthermore, the overweight information collection system also includes: weighing detection equipment. The central control device can be connected to the weighing detection equipment (weighing instrument) deployed on the inspection lane. The central control device obtains the weighing data of the vehicle to be inspected from the weighing instrument via the network and uploads the weighing data along with images of the vehicle's front, body, and rear to the business system. The business system performs weight overweight detection on the vehicle based on the vehicle's weighing data and the corresponding limit standards.

[0072] Optionally, the over-limit information collection system further includes: a vehicle dimension collection device. The central control device can be connected to the vehicle dimension collection device. The central control device obtains the dimension data of the vehicle to be inspected from the vehicle dimension collection device via the network, and uploads the dimension data together with the vehicle front image, vehicle body image, vehicle rear image, weighing data, etc. to the business system. The vehicle dimension data includes the overall length, height, and width of the vehicle after loading. In some embodiments, the over-limit detection of the vehicle may include weight over-limit detection and dimension over-limit detection. The weighing detection device and the vehicle dimension collection device are connected to the central control device. When the vehicle passes through the detection area, the weighing detection device and the vehicle dimension collection device automatically collect relevant weighing data and dimension data. In step 250, the weighing data and dimension data are reported to the business system together with other data. The business system performs weight over-limit detection and dimension over-limit detection on the vehicle based on the vehicle's weighing data, dimension data, and corresponding limit standards, and handles over-limit vehicles.

[0073] Furthermore, toll booths currently charge truck tolls based on axle type, requiring manual verification of the vehicle's axle type or identification of the vehicle's axle number using an ETC tag. This embodiment accurately identifies the vehicle's axle number based on a vehicle image and pushes it along with other data to the business system, enabling more accurate vehicle detection and toll collection.

[0074] Optionally, the over-limit information collection method provided in this embodiment further includes: using a target detection model to perform target detection on the vehicle body image to obtain one or more detection target information, each detection target information including the category and confidence level of the detection target; then, determining the detection targets whose category is tire and whose confidence level is greater than a second preset threshold from the one or more detection target information, and determining the number of detection targets that meet the above requirements, and determining the number of detection targets that meet the requirements as the number of axles of the vehicle to be detected. After determining the number of axles of the vehicle to be detected, in step 250, the vehicle's axle number information is pushed to the business system along with other images and license plate information.

[0075] To sum up, the overload information collection method provided in this embodiment can solve the problem in the prior art that the recognition results cannot be associated when multiple independent cameras respectively identify the front, rear and body features of the vehicle. Since the shooting angles of the three cameras set on the overload detection equipment in this application overlap, and by accurately tracking and distinguishing the vehicles in the video stream, the recognition results of the three cameras can be associated with the same vehicle, forming correct and complete vehicle data, which will not cause confusion in the retained evidence.

[0076] Based on the same inventive concept, the present application also provides an over-limit information collection device, see Figure 6 , the device comprises:

[0077] An acquisition module 310 is configured to acquire a first video stream captured by a front camera, a second video stream captured by a side camera, and a third video stream captured by a rear camera, provided on an overload detection device. The overload detection device is provided on one side of a detection lane, and the front camera, the side camera, and the rear camera are sequentially installed on the overload detection device in the direction of vehicle travel. The shooting angles of the front camera and the side camera, and the shooting angles of the side camera and the rear camera overlap.

[0078] The vehicle front recognition module 320 is used to control the vehicle front camera to capture a front image of the vehicle to be detected and identify the license plate information of the vehicle to be detected after detecting that the vehicle to be detected enters the first video stream;

[0079] a side recognition module 330 for tracking the vehicle to be detected in the second video stream and obtaining a vehicle body image of the vehicle to be detected based on the second video stream;

[0080] The vehicle rear recognition module 340 is configured to control the vehicle rear camera to capture a rear image of the vehicle to be detected and identify the rear license plate information of the vehicle to be detected after detecting that the vehicle to be detected in the second video stream has traveled to a preset area in the video image;

[0081] The information uploading module 350 is used to upload the vehicle front image, vehicle body image, vehicle rear image, front license plate information and rear license plate information corresponding to the vehicle to be detected to the business system.

[0082] Optionally, the side camera is a fisheye camera, and the side recognition module 330 includes an image stitching submodule, which is used to extract multiple frames of vehicle body side images from the second video stream, and stitch the multiple frames of vehicle body side images into one image through image stitching to obtain the vehicle body picture.

[0083] Optionally, the device also includes: a vehicle detection module, which is used to perform target detection on the first video stream using a target detection model to obtain one or more detection target information, wherein the detection target information includes the category and confidence of the detection target; and determines whether there is a detection target in the one or more detection target information whose category is a preset vehicle category and the corresponding confidence is greater than a first preset threshold. If so, it is determined that a vehicle to be detected has entered the first video stream.

[0084] Optionally, the device also includes: an axle detection module, which is used to perform target detection on the vehicle body image using a target detection model to obtain one or more detection target information, wherein the detection target information includes the category and confidence of the detection target; determine the detection target whose category is tire and whose confidence is greater than a second preset threshold from the one or more detection target information, and determine the number of detection targets that meet the above requirements, and determine the number as the number of axles of the vehicle to be detected.

[0085] Optionally, the device also includes: a video capture module, used to capture a vehicle entry video of a preset length from the first video stream, the second video stream or the third video stream, and upload the vehicle entry video together with the vehicle front picture, vehicle body picture, and vehicle rear picture to the business system.

[0086] The above-mentioned over-limit information collection device has the same basic principles and technical effects as the previous method embodiment. For the sake of brief description, for parts not mentioned in this embodiment, please refer to the corresponding content in the above-mentioned method embodiment, and no further details will be given here.

[0087] Figure 7 A possible structure of the central control device 400 provided in the embodiment of the present application is shown. Figure 7 The central control device 400 includes a processor 410, a memory 420, and a communication interface 430. These components are interconnected and communicate with each other through a communication bus 440 and / or other forms of connection mechanisms (not shown).

[0088] The memory 420 includes one or more (only one is shown in the figure), which may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The processor 410 and other possible components can access the memory 420 and read and / or write data therein.

[0089] The processor 410 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 410 can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0090] Communication interface 430 includes one or more (only one is shown in the figure) interfaces that can be used to communicate directly or indirectly with other devices to exchange data. Communication interface 430 can be an Ethernet interface; a mobile communication network interface, such as a 3G, 4G, or 5G network interface; or other types of interfaces that have data transmission and reception capabilities.

[0091] One or more computer program instructions may be stored in the memory 420 , and the processor 410 may read and execute these computer program instructions to implement the steps of the over-limit information collection method provided in the embodiment of the present application and other desired functions.

[0092] I understand. Figure 7 The structure shown is for illustration only. The central control device 400 may also include Figure 7 More or fewer components than shown, or with Figure 7 Different configurations shown. Figure 7 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0093] The present application also provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are read and executed by a computer processor, the steps of the over-limit information collection method provided in the present application are executed. For example, the computer-readable storage medium can be implemented as Figure 7 The memory 420 in the central control device 400.

[0094] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the over-limit information collection method provided by the embodiment of the present application.

[0095] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0096] Furthermore, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0097] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0098] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for collecting over-limit information, characterized in that: The method comprises: Obtain a first video stream captured by a front camera, a second video stream captured by a side camera, and a third video stream captured by a rear camera, which are arranged on an over-limit detection device; the over-limit detection device is arranged on one side of a detection lane, and the front camera, the side camera, and the rear camera are sequentially installed on the over-limit detection device according to the direction of vehicle travel, and there is overlap between the shooting angles of the front camera and the side cameras, and between the shooting angles of the side cameras and the rear camera; the side cameras are fisheye cameras; the over-limit detection device includes a column; a plurality of limiting holes are provided on a supporting bottom surface of the column; the over-limit detection device is installed using the plurality of limiting holes and a plurality of matching screws; After detecting that a vehicle to be detected enters the first video stream, controlling the front camera to capture a front view of the vehicle to be detected and identifying the front license plate information of the vehicle to be detected; Tracking the vehicle to be detected in the second video stream, and obtaining a vehicle body image of the vehicle to be detected based on the second video stream; After detecting that the vehicle to be detected in the second video stream drives into a preset area in the video screen, controlling the rear camera to capture a rear image of the vehicle to be detected and identifying the rear license plate information of the vehicle to be detected; Upload the vehicle front image, vehicle body image, vehicle rear image, front license plate information and rear license plate information corresponding to the vehicle to be detected to the business system; The method further comprises: A vehicle entry video of a preset length is captured from the first video stream, the second video stream or the third video stream, and the vehicle entry video is uploaded to the business system together with a vehicle front image, a vehicle body image and a vehicle rear image.

2. The method according to claim 1, characterized in that The obtaining of a vehicle body image of the vehicle to be detected based on the second video stream includes: A plurality of frames of vehicle body side images are extracted from the second video stream, and the plurality of frames of vehicle body side images are spliced ​​into one image by image stitching to obtain the vehicle body picture.

3. The method according to claim 1 or 2, characterized in that After obtaining the first video stream captured by the vehicle front camera provided on the over-limit detection device, the method further includes: Performing target detection on the first video stream using a target detection model to obtain one or more pieces of detected target information, the detected target information including a category and a confidence level of the detected target; Determine whether there is a detection target in the one or more detection target information whose category is a preset vehicle category and whose corresponding confidence is greater than a first preset threshold. If so, determine that a vehicle to be detected has entered the first video stream.

4. The method according to claim 1 or 2, characterized in that The method further comprises: Performing target detection on the vehicle body image using a target detection model to obtain one or more pieces of detection target information, wherein the detection target information includes a category and a confidence level of the detection target; Determine from the one or more detection target information the detection targets whose category is tire and whose confidence is greater than a second preset threshold, and determine the number of detection targets that meet the above requirements, and determine the number as the number of axles of the vehicle to be detected.

5. The method according to claim 1, wherein The method further comprises: The weighing data of the vehicle to be inspected is obtained from the weighing instrument, and the weighing data is uploaded to the business system together with the vehicle front picture, vehicle body picture, and vehicle rear picture.

6. An over-limit detection device, characterized in that: include: A plurality of limit holes are provided on the supporting bottom surface of the column; the over-limit detection device is installed through the plurality of limit holes and a plurality of matching screws; an over-limit detection device provided on the pillar; the over-limit detection device comprising a front camera, a side camera, a rear camera, a controller, and a switch, the front camera, the side camera, and the rear camera being respectively connected to the controller, which is connected to the switch; the front camera, the side camera, and the rear camera being sequentially installed on the side of the pillar in a preset direction, with the shooting angles of the front camera and the side camera overlapping, and the shooting angles of the side camera and the rear camera overlapping; the side cameras being fisheye cameras; The front camera is used to capture the first video stream and take a snapshot of the front of the vehicle to be detected; the side camera is used to capture the second video stream and obtain a body image of the vehicle to be detected based on the second video stream; the rear camera is used to capture the third video stream and take a snapshot of the rear of the vehicle to be detected; a vehicle entry video of a preset length is intercepted from the first video stream, the second video stream or the third video stream.

7. The device according to claim 6, characterized in that The pillar has a plurality of side surfaces, and the front camera, the side camera and the rear camera are respectively installed on one of the plurality of side surfaces.

8. An over-limit information collection device, characterized in that: The device comprises: An acquisition module is used to acquire a first video stream captured by a front camera, a second video stream captured by a side camera, and a third video stream captured by a rear camera, which are arranged on an over-limit detection device; the over-limit detection device is arranged on one side of a detection lane, and the front camera, the side camera, and the rear camera are sequentially installed on the over-limit detection device according to the direction of vehicle travel, and there is overlap between the shooting angles of the front camera and the side camera, and between the shooting angles of the side camera and the rear camera; the side camera is a fisheye camera; the over-limit detection device includes a column; a plurality of limiting holes are provided on the supporting bottom surface of the column; the over-limit detection device is installed using the plurality of limiting holes and a plurality of matching screws; A vehicle front recognition module is used to control the vehicle front camera to capture a front image of the vehicle to be detected and identify the license plate information of the vehicle to be detected after detecting that the vehicle to be detected enters the first video stream; a side recognition module, configured to track the vehicle to be detected in the second video stream and obtain a vehicle body image of the vehicle to be detected based on the second video stream; A rear vehicle recognition module is configured to control the rear camera to capture a rear image of the vehicle to be detected and identify the rear license plate information of the vehicle to be detected after detecting that the vehicle to be detected in the second video stream has traveled to a preset area in the video image; An information uploading module is used to upload the vehicle front image, vehicle body image, vehicle rear image, front license plate information and rear license plate information corresponding to the vehicle to be detected to the business system; The device also includes: a video capture module, which is used to capture a vehicle entry video of a preset length from the first video stream, the second video stream or the third video stream, and upload the vehicle entry video together with the vehicle front image, vehicle body image, and vehicle rear image to the business system.

9. An over-limit information collection system, characterized in that: include: The over-limit detection device and central control device according to claim 6 or 7, wherein the over-limit detection device is communicatively connected to the central control device; The over-limit detection device is used to send a first video stream captured by a front camera, a second video stream captured by a side camera, and a third video stream captured by a rear camera, which are arranged on the over-limit detection device, to the central control device; The central control device is used to execute the over-limit information collection method as described in any one of claims 1-5.

Images