A vehicle attribute tracking method, system, device and medium under a panoramic environment

Abstract

The application provides a vehicle attribute tracking method, system, device and medium under a surround view, the method comprising: setting a plurality of image acquisition units in the circumferential direction of a vehicle body, acquiring vehicle body outside images through the image acquisition units; performing target detection according to the vehicle body outside images to obtain target objects in each vehicle body outside image; determining an image sequence of the same target object according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units; and determining the speed and distance of the corresponding target object from the vehicle body according to the image sequence. The application uses multiple image acquisition units to cooperatively track targets, which can effectively ensure the accuracy of tracking detection of the same target object.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of driving safety applications, and in particular to a vehicle attribute tracking method, system, device and medium under a panoramic view environment. BACKGROUND

[0002] In recent years, with the continuous improvement of people's living standards, the number of motor vehicles is increasing, and more and more families use cars as a means of transportation. Although cars can greatly meet people's travel needs, but the road traffic conditions are getting worse, and more seriously, the phenomenon of vicious traffic accidents and endangering traffic safety is getting worse. Car collision events occur from time to time, and more and more car owners pay attention to travel safety. When a car collides with a pedestrian, or a rear-end collision occurs between the front and rear cars, the public security department mainly relies on on-site observation and road monitoring video to determine the accident liability, and the vehicle-mounted driving recorder can also play a certain role in determining the accident liability. However, the monitoring camera and the driving recorder have some blind spots that cannot be observed, and the video collected by them can only qualitatively present the process of the accident, lacking accurate quantitative data analysis. When law enforcement personnel determine the accident liability based on these video recordings, they often need to spend a lot of time watching repeatedly, and even analyze frame by frame, which requires law enforcement personnel to have a high level of professional skills. In addition, the existing distance measuring and speed measuring methods are based on a single camera for estimation, and the error of the estimation based on a single camera is relatively large. SUMMARY

[0003] The purpose of the present application is to provide a vehicle attribute tracking method, system, device and medium under a panoramic view environment, to solve the problem of poor accuracy of vehicle tracking and distance measurement in the prior art.

[0004] In order to achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0005] The present application provides a vehicle attribute tracking method under a panoramic view environment, comprising:

[0006] A plurality of image acquisition units are arranged in the circumferential direction of the vehicle body, and the image acquisition units are used to acquire images outside the vehicle body;

[0007] Target detection is performed according to the images outside the vehicle body, and target objects in each image outside the vehicle body are obtained;

[0008] The image sequence of the same target object is determined according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units;

[0009] The speed of the corresponding target object and the distance from the vehicle body are determined according to the image sequence.

[0010] In an embodiment of the present application, target detection is performed according to the vehicle exterior image to obtain target objects in each vehicle exterior image, including:

[0011] The image acquisition units are grouped according to their distribution on the vehicle body to obtain multiple camera groups;

[0012] Target detection is performed on the vehicle exterior images obtained by each camera group respectively to obtain target objects corresponding to each camera group.

[0013] In an embodiment of the present application, the image sequence of the same target object is determined according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units, including:

[0014] Similar image features are determined according to the intersection-over-union of the image features, and the similar image features are stored in a first feature sequence;

[0015] If the intersection of the image features is empty, the position difference between the image features is determined according to the acquisition time sequence, the image features corresponding to the same target object are determined according to the position difference, and a second feature sequence is obtained;

[0016] The image sequence is determined according to the union of the first feature sequence and the second feature sequence.

[0017] In an embodiment of the present application, the image features corresponding to the same target object are determined according to the position difference, including:

[0018] The length prediction value of the whole vehicle is determined according to the position difference;

[0019] If the length prediction value is within a preset length range, the corresponding image features are associated as the image features of the same target object.

[0020] In an embodiment of the present application, the speed and distance of the target object from the vehicle body are determined according to the image sequence, including:

[0021] The speed of the corresponding target object is determined according to the position of each image in the image sequence in the field of view of the corresponding image acquisition unit and the center interval of the field of view of adjacent image acquisition units;

[0022] The distance of the target object from the vehicle body is determined according to the proportion and position of the target object in the field of view of the image acquisition unit.

[0023] The present application also provides a vehicle attribute tracking system in a surround view environment, including:

[0024] An image acquisition module is configured to set a plurality of image acquisition units in the circumferential direction of the vehicle body, and acquire vehicle body outside images through the image acquisition units;

[0025] A target detection module is configured to perform target detection according to the vehicle body outside images, and obtain target objects in each vehicle body outside image;

[0026] A target tracking module is configured to determine an image sequence of the same target object according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units;

[0027] A speed / distance measurement module is configured to determine the speed and distance of the corresponding target object from the vehicle body according to the image sequence.

[0028] In an embodiment of the present application, the target detection module includes a plurality of target trackers, each of which corresponds to a plurality of image acquisition units, and is configured to label a graphic frame of a target object in a vehicle body outside image acquired by the corresponding image acquisition unit.

[0029] The present application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that the processor implements the steps of the vehicle attribute tracking method in the all-around view environment when executing the computer program.

[0030] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the vehicle attribute tracking method in the all-around view environment when executed by a processor.

[0031] The present application has the following beneficial effects:

[0032] The present application sets a plurality of image acquisition units in the circumferential direction of the vehicle body, acquires vehicle body outside images through the image acquisition units, performs target detection according to the vehicle body outside images, obtains target objects in each vehicle body outside image, determines an image sequence of the same target object according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units, and determines the speed and distance of the corresponding target object from the vehicle body according to the image sequence. The present application cooperates with a plurality of image acquisition units to track targets, and can effectively ensure the accuracy of tracking detection of the same target object. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 FIG. 1 is a flowchart of the vehicle attribute tracking method in the all-around view environment according to an embodiment of the present application.

[0034] Figure 2 FIG. 2 is a module diagram of the vehicle attribute tracking system in the all-around view environment according to an embodiment of the present application.

[0035] Figure 3 A schematic diagram of the structure of the device in an embodiment of the present application.

[0036] Figure 4 A schematic diagram of the structure of the device in another embodiment of the present application. DETAILED DESCRIPTION

[0037] The other advantages and effects of the present application can be easily understood by those skilled in the art from the disclosure of the present specification. The present application can also be implemented or applied by means of other different specific embodiments, and the details in the present specification can be modified or changed in various ways based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, but not for limiting the protection scope of the present application.

[0038] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, but not drawn according to the number, shape and size of the components in actual implementation. The shape, number and proportion of each component in actual implementation can be arbitrarily changed, and the layout pattern of the components can also be more complex.

[0039] Please refer to Figure 1 The present application provides a vehicle attribute tracking method in a surround view environment, which comprises the following steps.

[0040] Step S01, a plurality of image acquisition units are arranged on the vehicle body in the axial direction, and the vehicle body outside images are acquired by the image acquisition units.

[0041] Please refer to Figure 2 , Figure 2 A schematic diagram of the distribution of the image acquisition units in the circumferential direction of the vehicle in an embodiment of the present application. In an embodiment, as shown in Figure 2 , six image acquisition units, camera1-front, camera2-front-right, camera3-rear-right, camera4-rear, camera5-rear-left and camera6-front-left, can be installed on the vehicle body. The field of view angle of each image acquisition unit can be set to 90 degrees. The number of specific image acquisition units and the angle of the field of view angle can be adjusted according to the actual application requirements, which are not limited here. The vehicle body outside images in the corresponding field of view angle during driving are acquired by each image acquisition unit, so as to analyze the target tracking of the acquired images.

[0042] Step S02, target detection is performed according to the vehicle body outer side image to obtain a target object in each vehicle body outer side image.

[0043] In an embodiment, the target detection according to the vehicle body outer side image to obtain a target object in each vehicle body outer side image comprises:

[0044] According to the distribution of the image acquisition units on the vehicle body, the image acquisition units are grouped to obtain a plurality of camera groups.

[0045] Target detection is performed on the vehicle body outer side image obtained by each camera group to obtain a target object corresponding to each camera group.

[0046] In an embodiment, as shown in Figure 2 The image acquisition units distributed in the axial direction of the vehicle body are grouped, and each group processes the image information obtained by the image acquisition units in the group. This reduces the processing amount and improves the processing efficiency. For example, camera1-front, camera3-rear-right, and camera5-rear-left form a camera group, and camera2-front-right, camera4-rear, and camera6-front-left form another camera group. A tracker tracker-A and a tracker tracker-B are respectively assigned to the two camera groups. The trackers perform target detection on the images collected by the respective camera groups, label the shape boxes of the target objects in the images, and generate target object ID numbers, etc. The specific grouping can be adjusted according to actual application requirements, which is not limited here.

[0047] Step S03, according to the collection time sequence and image features of the target objects in adjacent image acquisition units, the image sequence of the same target object is determined.

[0048] In an embodiment, according to the collection time sequence and image features of the target objects in adjacent image acquisition units, the image sequence of the same target object is determined, comprising:

[0049] According to the intersection-over-union between each image feature, similar image features are determined, and the similar image features are stored in a first feature sequence;

[0050] If the intersection between each image feature is an empty set, then the position difference between each image feature is determined according to the collection time sequence, and the image features corresponding to the same target object are determined according to the position difference to obtain a second feature sequence;

[0051] The image sequence is determined according to the union of the first feature sequence and the second feature sequence.

[0052] In an embodiment, the image features corresponding to the same target object are determined according to the position difference, comprising:

[0053] A length prediction value of the whole vehicle is determined according to the position difference;

[0054] If the length prediction value is within a preset length range, the corresponding image features are associated as image features of the same target object.

[0055] In an embodiment, the region where the target object is located in the collected image is determined according to the target detection result in step S02, the corresponding region is marked by a frame, and the image features of the marked region can be further extracted by a convolutional neural network. First, it is determined whether the target objects obtained by the two adjacent image collection units exist in intersection. If there is intersection, it is determined whether the corresponding regions correspond to the same target object according to the intersection-union ratio of the images in the corresponding regions. If the intersection-union ratio reaches a preset threshold, it is considered that the target objects in the two images are the same target object. If there is no intersection or the intersection-union ratio is less than the set threshold, it can be determined whether the image features across the two field angles correspond to the same target object by the position difference between the image features corresponding to the images in the two adjacent image collection units at the same time node. For example, vehicle A comes from the right rear of the current vehicle, and vehicle A first enters the field angle of camera4-rear, at which time the image region of the left front head of vehicle A can be obtained, and if the field angle is sufficient, the image of the left side region of vehicle A can also be obtained. As vehicle A moves forward, the left front head region of vehicle A enters the field range of camera3-rear-right, and only the image of the left rear of vehicle A can be obtained in camera4-rear. In this case, if there is no overlap between the field ranges of camera3-rear-right and camera4-rear, the middle region of the vehicle will be blocked. The front wheel and rear wheel images can be obtained by the two image collection units. In order to avoid misjudgment, the position difference of the target object in the two image collection units can be calculated to further predict the length of the whole vehicle. For example, the distance between the front wheel and the rear wheel is used to predict the length of the whole vehicle, and it is determined whether the predicted length is within a set range. If it is within the preset length range, it is considered that the target objects corresponding to the two images are the same target object.

[0056] In an embodiment, there is another case, still taking the A vehicle as an example, the head region of the A vehicle enters the field of view of the camera3-rear-right at the t1 time node, the tail region of the A vehicle completely leaves the field of view of the camera3-rear-right at the t2 time node, and the head region of the A vehicle enters the field of view of the camera2-front-right at the t3 time node, where t3>t2>t1. In this case, the field of view of the camera3-rear-right and the field of view of the camera2-front-right are spaced apart enough, so that the position difference between the tail image and the head image at the t3 time node can be obtained through the time node between t1 and t2, the length of the vehicle is predicted, and it is determined whether the images collected by the two image collection units correspond to the same target object. The images corresponding to the same target object are classified into an image sequence according to the collection time sequence.

[0057] In step S04, the speed of the target object and the distance from the vehicle body are determined according to the image sequence.

[0058] In an embodiment, the speed of the target object and the distance from the vehicle body are determined according to the image sequence, including:

[0059] The speed of the target object is determined according to the position of each image in the image sequence in the field of view of the corresponding image collection unit and the center spacing of the field of view of the adjacent image collection unit.

[0060] The distance of the target object from the vehicle body is determined according to the proportion and position of the target object in the field of view of the image collection unit.

[0061] In an embodiment, since the positions and the field of view of the image collection units are relatively fixed, the center spacing of the field of view of the image collection units is also relatively determined, and the speed of the target object can be calculated through the position of each image in the image sequence in the field of view of the corresponding image collection unit and the image collection time node. The distance of the target object from the vehicle can be determined according to the proportion difference and the position relationship of the images in the image sequence in different field of view.

[0062] In an embodiment, the target object of the present application can be a vehicle or a pedestrian.

[0063] The embodiment provides a vehicle attribute tracking system in a panoramic environment, which is used for executing the vehicle attribute tracking method in the panoramic environment as described in the foregoing method embodiment. Since the technical principle of the system embodiment is similar to that of the foregoing method embodiment, the same technical details are not repeatedly described herein.

[0064] Please refer to Figure 2 In an embodiment, the vehicle attribute tracking in the panoramic environment comprises: an image acquisition module 10, configured to set a plurality of image acquisition units in the circumferential direction of a vehicle body, and acquire vehicle body outside images through the image acquisition units.

[0065] A target detection module 11, configured to perform target detection according to the vehicle body outside images, and obtain target objects in each vehicle body outside image.

[0066] A target tracking module 12, configured to determine image sequences of the same target object according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units.

[0067] A speed / distance measurement module 13, configured to determine the speed and distance of the corresponding target object from the vehicle body according to the image sequences.

[0068] The embodiment of the present application further provides a device, which can comprise one or more processors and one or more machine readable media having instructions stored thereon, which when executed by the one or more processors, cause the device to perform the vehicle attribute tracking method in the panoramic environment. In actual application, the device can be a terminal device or a server. Examples of the terminal device can include a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, a vehicle-mounted computer, a desktop computer, a set-top box, a smart television, a wearable device, and the like. The embodiment of the present application is not limited to a specific device.

[0069] The embodiment of the present application further provides a non-volatile readable storage medium, which stores one or more programs. When the one or more programs are applied to a device, the device can execute instructions of steps of the vehicle attribute tracking method in the panoramic environment.

[0070] Figure 3The hardware structure schematic diagram of the terminal device provided by an embodiment of the present application is shown in the figure. As shown in the figure, the terminal device can include an input device 1100, a first processor 1101, an output device 1102, a first memory 1103, and at least one communication bus 1104. The communication bus 1104 is used to realize the communication connection between the elements. The first memory 1103 can include a high-speed RAM memory, and can also include a non-volatile storage NVM, for example, at least one disk memory. Various programs can be stored in the first memory 1103, used to complete various processing functions and realize the method steps of the embodiment.

[0071] Optionally, the first processor 1101 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic elements, which is coupled to the input device 1100 and the output device 1102 through wired or wireless connection.

[0072] Optionally, the input device 1100 can include various input devices, for example, at least one of a user interface facing a user, a device interface facing a device, a programmable interface of software, a camera, and a sensor. Optionally, the device interface facing a device can be a wired interface for data transmission between devices, and can also be a hardware insertion interface (for example, a USB interface, a serial port, etc.) for data transmission between devices. Optionally, the user interface facing a user can be, for example, a control button facing a user, a voice input device for receiving voice input, and a touch sensing device (for example, a touch screen with touch sensing function, a touchpad, etc.) for receiving user touch input. Optionally, the programmable interface of software can be, for example, an entrance for a user to edit or modify a program, for example, an input pin interface or an input interface of a chip, etc. The output device 1102 can include a display, a sound device, etc.

[0073] In the embodiment, the processor of the terminal device includes functions for executing each module of the voice recognition device in each device. The specific functions and technical effects can refer to the above-mentioned embodiments, and will not be described here.

[0074] Figure 4 The hardware structure schematic diagram of the terminal device provided by another embodiment of the present application is shown in the figure. Figure 4 is to Figure 3 In one specific embodiment in the implementation process. As shown in the figure, the terminal device of the embodiment can include a second processor 1201 and a second memory 1202.

[0075] The second processor 1201 executes computer program codes stored in the second memory 1202 to implement the above-described embodiments Figure 1 The method.

[0076] The second memory 1202 is configured to store various types of data to support the operation of the terminal device. Examples of these data include instructions for any application or method operating on the terminal device, such as messages, pictures, videos, etc. The second memory 1202 can contain random access memory (RAM) and can also include non-volatile memory, such as at least one disk memory.

[0077] Optionally, the first processor 1201 is disposed in the processing component 1200. The terminal device can further include a communication component 1203, a power supply component 1204, a multimedia component 1205, a voice component 1206, an input / output interface 1207, and / or a sensor component 1208. The terminal device specifically contains components, etc. according to the actual demand setting, and the embodiment does not limit this.

[0078] The processing component 1200 generally controls the overall operation of the terminal device. The processing component 1200 can include one or more second processors 1201 to execute instructions to complete all or part of the steps of the above-described Figure 1 The method. In addition, the processing component 1200 can include one or more modules to facilitate interaction between the processing component 1200 and other components. For example, the processing component 1200 can include a multimedia module to facilitate the interaction between the multimedia component 1205 and the processing component 1200.

[0079] The power supply component 1204 provides power for various components of the terminal device. The power supply component 1204 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing power for the terminal device.

[0080] The multimedia component 1205 includes a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen can include a liquid crystal display (LCD) and a touch panel (TP). If the display screen includes a touch panel, the display screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes and gestures on the touch panel. The touch sensor can not only sense the boundary of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe operation.

[0081] The voice component 1206 is configured to output and / or input voice signals. For example, the voice component 1206 includes a microphone (MIC) configured to receive external voice signals when the terminal device is in an operating mode, such as a voice recognition mode. The received voice signals may be further stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the voice component 1206 also includes a speaker for outputting voice signals.

[0082] Input / output interface 1207 provides an interface between processing component 1200 and peripheral interface modules, such as click wheels, buttons, etc. These buttons may include, but are not limited to, volume buttons, start buttons, and lock buttons.

[0083] Sensor assembly 1208 includes one or more sensors for providing status assessments of various aspects of the terminal device. For example, sensor assembly 1208 can detect the on / off state of the terminal device, the relative positioning of components, and the presence or absence of user contact with the terminal device. Sensor assembly 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, sensor assembly 1208 may also include a camera, etc.

[0084] Communication component 1203 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one embodiment, the terminal device may include a SIM card slot for inserting a SIM card, enabling the terminal device to log in to a GPRS network and establish communication with a server via the Internet.

[0085] As can be seen from the above, in Figure 4 The communication component 1203, voice component 1206, input / output interface 1207, and sensor component 1208 involved in the embodiment can all be used as... Figure 3 The implementation method of the input device in the embodiment.

[0086] The above embodiments are merely preferred embodiments provided to fully illustrate this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. A vehicle attribute tracking method in a surround view environment, characterized by, The method comprises the following steps: a plurality of image acquisition units are arranged in the circumferential direction of the vehicle body, and vehicle body outside images are acquired through the image acquisition units; target detection is performed according to the vehicle body outside images, and target objects in each vehicle body outside image are obtained; image sequences of the same target object are determined according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units; the image sequences of the same target object are determined according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units, which comprises the following steps: similar image features are determined according to the intersection and union ratio between each image feature, and the similar image features are stored in a first feature sequence; if the intersection between each image feature is an empty set, then the position difference between each image feature is determined according to the acquisition time sequence, the image features corresponding to the same target object are determined according to the position difference, and a second feature sequence is obtained; the image sequences are determined according to the union set of the first feature sequence and the second feature sequence; the speed of the corresponding target object and the distance from the vehicle body are determined according to the image sequences.

2. The vehicle attribute tracking method in a surround view environment according to claim 1, characterized by, The target detection is performed according to the vehicle body outside images, and the target objects in each vehicle body outside image are obtained, which comprises the following steps: the image acquisition units are grouped according to the distribution of the image acquisition units on the vehicle body, and a plurality of camera groups are obtained; target detection is performed on the vehicle body outside images obtained by each camera group respectively, and the target objects corresponding to each camera group are obtained.

3. The vehicle attribute tracking method in a surround view environment according to claim 1, characterized by, The image features corresponding to the same target object are determined according to the position difference, which comprises the following steps: a length prediction value of the whole vehicle is determined according to the position difference; if the length prediction value is within a preset length range, then the corresponding image features are associated as the image features of the same target object.

4. The vehicle attribute tracking method in a surround view environment according to claim 1, characterized by, The speed of the corresponding target object and the distance from the vehicle body are determined according to the image sequences, which comprises the following steps: the speed of the corresponding target object is determined according to the position of each image in the field of view angle of the corresponding image acquisition unit and the center interval of the field of view angles of adjacent image acquisition units; the distance between the target object and the vehicle body is determined according to the proportion and position of the target object in the field of view angle of the image acquisition unit.

5. A vehicle attribute tracking system under a surround view, characterized by, The method comprises the following steps: an image acquisition module is arranged in the circumferential direction of the vehicle body, and vehicle body outside images are acquired through the image acquisition units; a target detection module is used to perform target detection according to the vehicle body outside images, and target objects in each vehicle body outside image are obtained; a target tracking module is used to determine image sequences of the same target object according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units; the image sequences of the same target object are determined according to the acquisition time sequence and image features of the target objects in adjacent image acquisition units, which comprises the following steps: similar image features are determined according to the intersection and union ratio between each image feature, and the similar image features are stored in a first feature sequence; If the intersection between the image features is empty, a position difference between the image features is determined according to the acquisition time sequence, a second feature sequence of image features corresponding to the same target object is determined according to the position difference, and the image sequence is determined according to the union of the first feature sequence and the second feature sequence. A speed / distance measuring module is configured to determine the speed of the target object and the distance from the vehicle body according to the image sequence.

6. The vehicle attribute tracking system under the surround view according to claim 5, wherein, The target detection module includes a plurality of target trackers, each of which corresponds to a plurality of image acquisition units and is configured to mark the graphic frame of the target object in the image outside the vehicle body acquired by the corresponding image acquisition unit.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 4 when executing the computer program.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 4.

Images