Target object perception method and device, perception base station, and perception system
By dynamically switching image processing modes based on recognition accuracy in intelligent traffic and using multi-perception base station information sharing, the problem of high computing resource occupation in target object perception process in intelligent traffic is solved, the perception real-time and accuracy are improved, and the perception is reduced System power consumption.
Patent Information
- Application Number
- CN202010023297.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-01-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-01-09
AI Technical Summary
In intelligent transportation, the existing technology relies on frame-by-frame processing on real-time perception of target objects (such as vehicles and pedestrians), resulting in high computing resource usage, insufficient real-time performance and increased power consumption, making it difficult to meet the needs of efficient identification and tracking.
By acquiring the target image in the perception area, dynamically switch the processing mode according to the recognition accuracy, frame-by-frame processing and frame-skipping processing are adopted, combined with information sharing and linkage of multiple perception base stations, image acquisition and processing strategies are optimized, and computing resource requirements are reduced.
It improves the real-time and accuracy of the perceived target object, reduces data processing volume, reduces system power consumption, reduces hardware configuration requirements, and achieves more efficient intelligent traffic perception.
Smart Images

Figure CN113111682B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of intelligent transportation technology, and in particular to a target object perception method and device, a perception base station, and a perception system. Background Art
[0002] In intelligent transportation, it is necessary to perceive the target object (for example, vehicles, pedestrians, etc.) in real time. The perception of the target object mainly relies on one or more perception base stations. A perception module is installed on the perception base station. For example, the perception module is an image acquisition device. By acquiring the image of the target object and processing the image, the target object can be perceived. In order to ensure the perception accuracy, the collected images are often processed frame by frame, resulting in more occupied processing resources. Summary of the invention
[0003] The present disclosure provides a target object perception method and device, a perception base station, and a perception system.
[0004] According to a first aspect of an embodiment of the present disclosure, a target object perception method is provided, the method comprising: acquiring a target image including a target object within a perception area; determining a recognition accuracy for identifying identity identification information of the target object, and determining a processing mode for processing the target image according to the recognition accuracy; wherein the processing mode comprises a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image; the target image is processed based on the processing mode to obtain a perception result of the target object.
[0005] In some embodiments, determining the processing mode for processing the target image based on the recognition accuracy rate includes: if the recognition accuracy rate is lower than a preset recognition accuracy rate threshold, determining that the processing mode is the first processing mode; and / or if the recognition accuracy rate is not lower than the recognition accuracy rate threshold, determining that the processing mode is the second processing mode.
[0006] In some embodiments, the method is executed by a first perception base station in a perception link, and the perception link includes multiple perception base stations including the first perception base station; the method also includes: obtaining first characteristic information of the target object sent by a previous perception base station of the first perception base station and second characteristic information of the target object in the first perception base station; identifying the identity information of the target object based on the first characteristic information and the second characteristic information.
[0007] In some embodiments, the method further includes: acquiring an original image of the perception area; and screening out a target image including the target object from the original image.
[0008] In some embodiments, screening out the target image including the target object from the original image includes: detecting the original image through a deep learning algorithm to detect the target image including the target object from the original image.
[0009] In some embodiments, the processing the target image based on the processing mode includes: when the distance between the target image and the target object is less than a preset distance threshold, processing the target image based on the processing mode.
[0010] In some embodiments, the method further includes: after processing the target image based on the processing mode to obtain a perception result of the target object, drawing a map of the perception area according to the perception result.
[0011] In some embodiments, the map includes at least any of the following: the number of lanes in the perception area, information about the lane where the target object is located, feature information of the target object on each lane, congestion information on each lane, and obstacle distribution information on each lane.
[0012] In some embodiments, the target image includes a first target image acquired when the target object enters the perception area, and a second target image acquired when the target object exits the perception area; the processing mode includes a processing mode for processing the first target image and a processing mode for processing the second target image; processing the target image based on the processing mode to obtain the perception result of the target object includes: processing the first target image based on the processing mode for processing the first target image to obtain a first processing result; processing the second target image based on the processing mode for processing the second target image to obtain a second processing result; obtaining the perception result of the target object according to the first processing result and the second processing result.
[0013] In some embodiments, the first target image is acquired by a first image acquisition device, and the second target image is acquired by a second image acquisition device.
[0014] In some embodiments, the acquiring includes a target image of a target object, including: when the target object enters an untrackable area between the first image acquisition device and the second image acquisition device, enabling the second image acquisition device to acquire the second target image.
[0015] In some embodiments, the method further includes: if the time for starting the second image acquisition device is less than a preset time threshold, determining that the processing mode for processing the second target image is the first processing mode.
[0016] In some embodiments, the method also includes: after determining that the processing mode for processing the second target image is the first processing mode, if the first processing result matches the second processing result, switching the processing mode for processing the second target image from the first processing mode to the second processing mode.
[0017] In some embodiments, if the first image acquisition device does not acquire a first target image including the target object, the second image acquisition device is disabled.
[0018] In some embodiments, the method further includes: if the first image acquisition device has not acquired the first target image including the target object at the current moment, and the first image acquisition device has acquired the first target image including the target object within a period of time before the current moment, enabling the second image acquisition device to obtain the second target image.
[0019] In some embodiments, the processing of the first target image includes: identifying the target object in the first region of interest in the first target image; and / or tracking the target object in the first region of interest in the first target image; and / or the processing of the second target image includes: identifying the target object in the second region of interest in the second target image; and / or tracking the target object in the second region of interest in the second target image.
[0020] In some embodiments, when a first condition is not met, the processing of the first target image includes tracking the target object in the first region of interest; the first condition is: the recognition accuracy of identifying the identity information of several previous target objects before the target object is greater than a preset recognition accuracy threshold; and / or when a second condition is not met, the processing of the second target image includes tracking the target object in the second region of interest; the second condition is: the recognition accuracy of identifying the identity information of the target object is greater than a preset recognition accuracy threshold.
[0021] In some embodiments, the target object is added to a target object list when entering a sensing area; the method further comprises: in response to the target object leaving a trackable area, refreshing the target object list.
[0022] In some embodiments, when a first condition is met, processing the target image includes tracking the target object in the target image and identifying the target object in the target image; the first condition includes at least any one of the following: the recognition accuracy is lower than a preset recognition accuracy threshold; and tracking of the target object fails, or the confidence of the identity information is lower than a preset confidence threshold.
[0023] In some embodiments, the method further includes: if the identity information of the target object is not detected, reallocating the identity information to the target object.
[0024] According to a second aspect of an embodiment of the present disclosure, a target object perception method is provided, the method comprising: acquiring a target image including a target object in a perception area; determining a processing mode for processing the target image according to a current detection event; wherein the processing mode comprises a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image;
[0025] The target image is processed based on the processing mode to obtain a perception result of the target object.
[0026] In some embodiments, the current detection event includes at least any one of the following: detecting object information of the target object, detecting a specific event.
[0027] In some embodiments, the object information includes at least any of the following: a moving speed and a type of the target object.
[0028] In some embodiments, the processing mode for processing the target image based on the current detection event includes: if the object information satisfies a third condition, determining that the processing mode is the first processing mode; and / or if the object information does not satisfy the third condition, determining that the processing mode is the second processing mode; wherein the third condition includes: the moving speed of the target object is greater than a preset speed threshold, and / or the type of the target object is a preset type.
[0029] In some embodiments, the specific event includes a traffic violation event and / or receiving a processing mode switching instruction.
[0030] In some embodiments, determining the processing mode for processing the target image based on the current detection event includes: if a fourth condition is met, determining that the processing mode is the first processing mode; and / or if the fourth condition is not met, determining that the processing mode is the second processing mode; wherein the fourth condition includes: detecting the violation event, and / or receiving a processing mode switching instruction to switch the processing mode to the first processing mode.
[0031] According to a third aspect of an embodiment of the present disclosure, a target object perception method is provided, the method comprising: determining a current detection event; determining a shooting mode for photographing a target object in a perception area according to the current detection event; wherein the shooting mode comprises a first shooting mode for photographing the target object at a first frame rate, and a second shooting mode for photographing the target object at a second frame rate, the first frame rate being greater than the second frame rate.
[0032] According to a fourth aspect of an embodiment of the present disclosure, a target object perception method is provided, the method comprising: acquiring a multi-frame target image including a target object within a perception area; determining a sending mode for sending the multi-frame target image to a processing unit for processing according to a current detection event; wherein the sending mode comprises a first sending mode for sending the multi-frame target image frame by frame, and a second sending mode for sending the multi-frame target image frame by frame.
[0033] In some embodiments, the method further includes: a frame skipping step size of frame skipping transmission of the multiple frames of target images is adjustable.
[0034] According to a fifth aspect of an embodiment of the present disclosure, a target object perception method is provided, the method comprising: respectively acquiring target images of multiple target objects in a perception area; processing the target image of each target object using different processing modes, wherein the processing modes include a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image, and the processing mode for processing the target image of each target object is determined according to a current detection event related to the target object.
[0035] According to a sixth aspect of an embodiment of the present disclosure, a target object perception device is provided, the device comprising: a first acquisition module, used to acquire a target image including a target object in a perception area; a first determination module, used to determine a recognition accuracy for identifying identity identification information of the target object, and determine a processing mode for processing the target image according to the recognition accuracy; wherein the processing mode comprises a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image; a first perception module, used to process the target image based on the processing mode to obtain a perception result of the target object.
[0036] According to the seventh aspect of an embodiment of the present disclosure, a target object perception device is provided, the device comprising: a second acquisition module, used to acquire a target image including a target object in a perception area; a second determination module, used to determine a processing mode for processing the target image according to a current detection event; wherein the processing mode comprises a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image; a second perception module, used to process the target image based on the processing mode to obtain a perception result of the target object.
[0037] According to an eighth aspect of an embodiment of the present disclosure, a target object perception device is provided, the device comprising: a third determination module for determining a current detection event; a fourth determination module for determining a shooting mode for photographing a target object in a perception area according to the current detection event; wherein the shooting mode comprises a first shooting mode for photographing the target object at a first frame rate, and a second shooting mode for photographing the target object at a second frame rate, the first frame rate being greater than the second frame rate.
[0038] According to a ninth aspect of an embodiment of the present disclosure, a target object perception device is provided, the device comprising: a third acquisition module, for acquiring a plurality of frames of target images including a target object within a perception area; a fifth determination module, for determining a sending mode for sending the plurality of frames of target images to a processing unit for processing according to a current detection event; wherein the sending mode comprises a first sending mode for sending the plurality of frames of target images frame by frame, and a second sending mode for sending the plurality of frames of target images frame by frame.
[0039] According to the tenth aspect of an embodiment of the present disclosure, a target object perception device is provided, the device comprising: a fourth acquisition module, for respectively acquiring target images of multiple target objects in a perception area; a third perception module, for processing the target image of each target object using different processing modes to respectively acquire perception results of each target object; wherein the processing modes comprise a first processing mode for performing frame-by-frame processing on the target image, and a second processing mode for performing frame skipping processing on the target image, and the processing mode for processing the target image of each target object is determined according to a current detection event related to the target object.
[0040] According to an eleventh aspect of an embodiment of the present disclosure, a perception base station is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment when executing the program.
[0041] According to a twelfth aspect of an embodiment of the present disclosure, a perception system is provided, and the perception system includes the perception base station described in any embodiment.
[0042] In some embodiments, the perception system further includes: an image acquisition device; the image acquisition device is used to acquire the target image.
[0043] In some embodiments, the image acquisition device includes: a first image acquisition device and a second image acquisition device; the first image acquisition device is used to acquire a first target image including the target object when the target object enters the perception area, and the second image acquisition device is used to acquire a second target image including the target object when the target object exits the perception area.
[0044] According to the thirteenth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any one of the embodiments when executing the program.
[0045] The disclosed embodiment obtains a target image including a target object in a perception area, determines the recognition accuracy of the identity information of the target object, determines a processing mode for processing the target image according to the recognition accuracy, and processes the target image based on the processing mode to obtain a perception result of the target object. The processing mode for processing the target image can be switched to a first processing mode for frame-by-frame processing or a second processing mode for frame skipping processing according to actual conditions. Compared with the method of processing the acquired target image frame by frame, the amount of data processing in the process of target object perception is reduced, and the real-time perception is improved.
[0046] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.
[0048] Figure 1 It is a schematic diagram of two different perception base stations performing perception linkage according to an embodiment of the present disclosure.
[0049] Figure 2 It is a functional schematic diagram of a perception base station according to an embodiment of the present disclosure.
[0050] Figure 3 It is a flow chart of the target object perception method of the embodiment of the present disclosure.
[0051] Figure 4 Schematic diagram of a non-trackable area according to an embodiment of the present disclosure.
[0052] Figure 5 It is a schematic diagram of target images captured by the image acquisition device at different times according to an embodiment of the present disclosure.
[0053] Figure 6 It is a program flow chart of the target object perception process of the embodiment of the present disclosure.
[0054] Figure 7 It is a schematic diagram of a map according to an embodiment of the present disclosure.
[0055] Figure 8 It is a flow chart of a target object perception method according to another embodiment of the present disclosure.
[0056] Fig. 9 It is a flow chart of a target object perception method according to another embodiment of the present disclosure.
[0057] Fig.10 It is a flow chart of a target object perception method according to another embodiment of the present disclosure.
[0058] Fig.11 It is a flow chart of a target object perception method according to another embodiment of the present disclosure.
[0059] Fig.12 It is an overall flow chart of image acquisition, image transmission and image processing of an embodiment of the present disclosure.
[0060] Fig.13 It is a block diagram of a target object perception device according to an embodiment of the present disclosure.
[0061] Fig.14It is a block diagram of a target object perception device according to another embodiment of the present disclosure.
[0062] Fig.15 It is a block diagram of a target object perception device according to another embodiment of the present disclosure.
[0063] Fig.16 It is a block diagram of a target object perception device according to another embodiment of the present disclosure.
[0064] Fig.17 It is a block diagram of a target object perception device according to another embodiment of the present disclosure.
[0065] Fig.18 It is a schematic diagram of a perception base station according to an embodiment of the present disclosure.
[0066] Fig.19 is a schematic diagram of a target object perception system according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0068] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.
[0069] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0070] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings.
[0071] In intelligent transportation, real-time tracking and identification of target objects (for example, vehicles) are mainly achieved by one or more sensing base stations on the sensing link. A sensing module is installed on the sensing base station. The sensing module includes the following functions: vehicle sensing of a single sensing base station (including vehicle tracking and vehicle identification), and sensing linkage between sensing base stations. Sensing linkage means that multiple sensing base stations perform joint sensing of vehicles (including joint vehicle tracking and joint vehicle identification) through sensing linkage and vehicle information sharing. Figure 1 As shown, it is a schematic diagram of perception linkage between two different perception base stations in some embodiments.
[0072] Generally speaking, there may be multiple sensing base stations inside a access service point (for example, a highway toll station). There are no sensing blind spots between the sensing base stations, and the sensing base stations can communicate with each other. By burying induction coils on the road surface at the boundary of the sensing base station or installing millimeter-wave radars, when the vehicle just enters the sensing area of the sensing base station, the boundary sensing base station is triggered to start detection. After that, within the entire access service point, all relevant sensing base stations can continuously identify and track the vehicle throughout the sensing area. Therefore, in the entire sensing area, the entire sensing system will have an accurate grasp of the vehicle's position. Among them, the determination of the vehicle's position can be completed through artificial intelligence computer vision algorithms, such as target tracking algorithms and target recognition algorithms. The functional schematic diagram of a single sensing base station is shown in the figure below. Figure 2 shown.
[0073] Furthermore, in the application scenario of smart transportation contactless payment, in order to support vehicles to pass efficiently without stopping, or even without slowing down, and at the same time meet the delay requirements of the payment scenario, a single perception base station is often required to perform frame-by-frame operations on each vehicle, and realize real-time vehicle identification and tracking. That is, almost all perception-related business logic must be processed frame by frame, such as timely identification and continuous tracking of vehicle license plates.
[0074] In the intelligent transportation service payment scenario, for vehicle identification, the validity and uniqueness of the vehicle's identity information (ID) in the entire perception link must be guaranteed, that is, each vehicle corresponds to a unique ID, and this ID will not be dynamically deleted unless the vehicle leaves the perception range. Therefore, the vehicle identification and tracking algorithm plays two roles: within the perception area of a single perception base station, it ensures that the vehicle with a determined ID continues to maintain its original identity; in the overlapping area of adjacent perception base stations, through perception linkage, it ensures that the vehicle ID remains unchanged and is not lost, and through perception linkage, the consistency of the vehicle identity is further maintained in time and space, further improving the accuracy of system detection.
[0075] Therefore, in order to ensure the uniqueness of the ID, the vehicle identification algorithm generally ensures the accuracy of identification through multi-point redundant detection, that is, if the vehicle is identified in the coverage area of one camera, it must be re-identified in the coverage area of another camera. However, in actual use, whether in the coverage area of one camera or in the coverage area of multiple cameras, the tracking algorithm has certain errors, resulting in potential target loss, and the loss of the target or the low recognition rate (confidence) of the tracking may lead to payment failure. Therefore, in order to ensure the stability of the ID, the vehicle must be detected and tracked again frame by frame, with multi-point redundancy and multi-point linkage.
[0076] However, the detection method of frame-by-frame detection, multi-point redundancy, and multi-camera linkage has high requirements for system calculations, which easily leads to insufficient real-time capabilities of the system. For example, to process 30 frames of images per second, each frame requires at least tens of milliseconds to process, resulting in low real-time processing performance; at the same time, the system power consumption increases, causing heat dissipation problems.
[0077] In view of the above problems, the embodiments of the present disclosure propose a target object perception solution, which provides a set of optimal real-time control methods for real-time vehicle tracking and identification through single-point detection, detection linkage, and information sharing. While improving the real-time and accuracy of target tracking, it reduces the requirements for computer hardware configuration and further reduces costs. Figure 3 As shown, a target object perception method according to an embodiment of the present disclosure is shown, and the method includes:
[0078] Step 302: Acquire a target image including a target object in a sensing area;
[0079] Step 304: determining a recognition accuracy rate for identifying the identity information of the target object, and determining a processing mode for processing the target image according to the recognition accuracy rate; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image;
[0080] Step 306: Process the target image based on the processing mode to obtain a perception result of the target object.
[0081] The method of the embodiment of the present disclosure may be performed by a perception base station (hereinafter referred to as base station A). The target object may be a mobile target such as a vehicle, an aircraft, or a pedestrian. The number of target objects in the perception area may change dynamically over time. At different times, the number of target objects in the same perception area may be 0, 1, or a value greater than 1. Each target may be perceived by a perception process in a perception base station, and multiple target objects may be perceived in parallel by multiple perception processes to improve perception efficiency.
[0082] The target image including the target object can be acquired through an image acquisition device, and the image acquisition device can be a camera, a camera, etc. The image acquisition device can acquire images of the image acquisition area including the perception area to obtain a video stream or an image frame. In some embodiments, the image acquisition device can be enabled to acquire the target image when the target object is about to enter the perception area of base station A. In the case where a plurality of perception base stations are included in the perception link, the perception result of the pre-perception base station on the target object (for example, the position and moving speed of the target object) can be acquired to determine whether the target object is about to enter the perception area of base station A, wherein the pre-perception base station of base station A is the previous perception base station of base station A.
[0083] The original image of the perception area captured by the image acquisition device may include the target object or may not include the target object. Therefore, it is necessary to filter the original image captured by the image acquisition device to filter out the target image including the target object from the original image. Specifically, the original image can be detected by a deep learning algorithm to detect the target image including the target object from the original image.
[0084] The image acquisition device can be installed on the perception base station, and the number of image acquisition devices on each perception base station can be one or more. In the case where the number of image acquisition devices is multiple, each image acquisition device can be used to acquire different target images. For example, one or some of the first image acquisition devices (referred to as the forward image acquisition device) can acquire a first target image including the target object when the target object enters the perception area; for another example, another or some of the second image acquisition devices (referred to as the backward image acquisition device) can acquire a second target image including the target object when the target object exits the perception area. Furthermore, the image acquisition device may also include a third image acquisition device (referred to as a blind image acquisition device) for acquiring images of the visual blind area between the forward image acquisition device and the backward image acquisition device. Whether the blind image acquisition device needs to be installed can be determined based on the scenario.
[0085] In some embodiments, it can be determined whether to enable the second image acquisition device based on actual conditions. For example, when the target object enters the untrackable area between the first image acquisition device and the second image acquisition device, the second image acquisition device is enabled to obtain the second target image. Among them, the untrackable area includes a perception blind area and a visual blind area. The perception blind area refers to an area within the visible area of the image acquisition device where the target object cannot be identified at a specific time or in a specific scene due to image processing algorithms or external environmental factors. The perception blind area may be a dynamically changing area. The visual blind area refers to an area that cannot be covered by the field of view of the image acquisition device, as well as an area that is within the field of view of the image acquisition device but is blocked by objects, resulting in the target object being unable to be captured by the image acquisition device. Figure 4 , is a schematic diagram of the non-trackable area of some embodiments of the present disclosure, wherein the white area is the field of view coverage area of the image acquisition device, the white area on the left is the field of view coverage area of the first image acquisition device, the white area on the right is the field of view coverage area of the second image acquisition device, the black area is the perception blind area, and the area between the field of view coverage areas of the two image acquisition devices (gray area) is the visual blind area. Both the visual blind area and the perception blind area are non-trackable areas.
[0086] For another example, if the first image acquisition device does not acquire the first target image including the target object, the second image acquisition device may be disabled. If the first image acquisition device does not acquire the first target image including any target object, it means that the target object may not exist in the sensing area. Therefore, the second image acquisition device may be disabled to reduce system power consumption and data processing. When the first image acquisition device re-acquires the first target image including the target object, the second image acquisition device is enabled.
[0087] For another example, if the first image acquisition device does not acquire the first target image including the target object at the current moment, and the first image acquisition device acquires the first target image including the target object within a period of time before the current moment, the second image acquisition device is enabled to acquire the second target image. Figure 5 As shown, at time t1, the first image acquisition device acquires a first target image including the target object A. After a period of △t, the target object A leaves the visual range of the first image acquisition device, and at this time, no other target object enters the visual range of the first image acquisition device. Although the first image acquisition device did not detect the existence of the target object at time t1+△t, since the first image acquisition device detected the existence of the target object at time t1, the target object actually exists in the perception area. Therefore, in this case, it is necessary to enable the second image acquisition device to perform joint perception of the target object. Joint perception refers to a method of comparing the perception results of multiple image acquisition devices, and then using information such as the speed and physical position of the target object to determine the perception result of the target object based on the differences in the perception results. Through joint perception, the loss of the target object can be reduced.
[0088] In addition, in the case of perception gaps, it is also possible that the first target image including the target object is not captured at the current moment, but the first target image including the target object is captured within a period of time before the current moment. The so-called perception gap refers to the instability of the perception result, which makes the number of perceived target objects vary from time to time. In this case, the perception result is not unique, which can easily cause the loss of the perceived target. In the case of perception gaps, the second image acquisition device can also be enabled to perform joint perception to improve the stability of the perception result.
[0089] Each target object may correspond to an identity information (ID) within the perception area, which is used to uniquely identify the target object. The identity information of the target object may be assigned to the target object when the target object enters the perception area. Each target object generally has unique identity information, and the identity information generally does not change before the target object leaves the perception area. After the target object leaves the perception area, the identity information of the target object may be deleted, and the deleted identity information may be reallocated to other target objects. In particular, if there are multiple perception base stations on a perception link, the identity information of the target object may be assigned to the target object by the perception base station (called a boundary perception base station) corresponding to the first perception area (called a boundary perception area) entered by the target object.
[0090] However, in actual situations, due to reasons such as the target object being blocked, when the target object enters the boundary perception area, the boundary perception base station may fail to assign identity information to the target object in time. Therefore, when the target object enters the perception area of other perception base stations, other perception base stations can reassign identity information to the target object. This process is called Re-ID. In addition, when two image acquisition devices perform joint perception of the target object, the perception results of the target object in the target image acquired by the two image acquisition devices will be shared. When the target object crosses the viewing area of two image acquisition devices, the perception results corresponding to the two image acquisition devices should be very close. For example, if the target object A leaves the visual area of the first image acquisition device, then theoretically the target object A should appear in the visual area of the second image acquisition device. However, when the target object A is in the visual blind spot between the two image acquisition devices, the perception results corresponding to the two image acquisition devices may not match (this situation is called the low degree of joint perception execution of multiple image acquisition devices). In this case, the target object also needs to be Re-ID.
[0091] When performing Re-ID, the identity information historically assigned to the target object can be searched based on the characteristic information of the target object. If found, the found identity information is assigned to the target object to maintain the uniqueness of the identity information of the target object in the entire perception link; if not found, the identity information is reassigned to the target object.
[0092] After acquiring the target image, the target object in the target image can be feature identified to obtain feature information of the target object, and then identity information is assigned to the target object based on the feature information. The feature information can be at least any one of the following: the model, color, size, character information on the target object, etc. In some embodiments, the target object is a vehicle, and accordingly, the feature information can be at least any one of the following: the model, color, size, license plate number, etc. of the vehicle. Since the process of capturing the target image may be affected by environmental factors, or the target object may be blocked by obstacles or other target objects, the acquired identity information is not always accurate. Therefore, some measures need to be taken to improve the recognition accuracy of the identity information of the target object.
[0093] In the case where a perception link includes multiple perception base stations, the first characteristic information of the target object sent by the previous perception base station of the first perception base station in the perception link and the second characteristic information of the target object in the first perception base station can also be obtained; the identity information of the target object is identified according to the first characteristic information and the second characteristic information. Through the linkage between multiple perception base stations, the recognition accuracy of identity information can be improved. The previous perception base station (base station B) of the first perception base station (base station A) can send the first characteristic information of the target object to base station A, and the first characteristic information carries the identity information of the target object. The sending method can be a broadcast method or other methods.
[0094] By sharing information among multiple sensing base stations, joint sensing of the target object is achieved, so that the target object has no blind spots during movement, thus improving the sensing accuracy. The information that can be shared during the joint sensing process includes, but is not limited to: character information on the target object, the color of the target object, the moving speed of the target object, the location of the target object (including the lane location and GPS location), etc. Furthermore, it may also include process data of various algorithms used in the sensing process.
[0095] Taking the broadcast mode as an example, base station A can read the nearby sensing base station target list periodically or according to the wireless broadcast frame address, identify the broadcast frame signal sent to its own address, and determine whether there is a target object in the sensing area of base station A according to the broadcast frame signal. If there is, base station A can read the characteristic information packet of the target object sensed by base station B, and compare it with the characteristic information packet of the target object sensed by itself to identify the identity information of the target object, thereby improving the recognition accuracy of the identity information.
[0096] When the number of pixels of the target object in the target image is less than a preset number threshold (e.g., 25×25), the recognition accuracy is less than the preset recognition accuracy threshold. When the number of pixels of the target object in the target image is not less than the number threshold, the recognition accuracy is not less than the recognition accuracy threshold.
[0097] If the recognition accuracy is lower than the preset recognition accuracy threshold, it can be determined that the processing mode for processing the target image is the first processing mode. By increasing the processing frame rate of the target image, the recognition accuracy of the identity information can be improved. Therefore, the processing mode can be switched to the first processing mode, and the target object in the target image is feature-recognized based on the first processing mode to obtain the identity information of the target object until the recognition accuracy of the identity information reaches the recognition accuracy threshold. If the recognition accuracy is not lower than the recognition accuracy threshold, it means that the vehicle has entered a monitoring area with stable and reliable perception. At this time, in order to reduce the system's computing power resource usage, reduce power consumption, and improve the system's real-time performance, the processing mode can be determined to be the second processing mode.
[0098] If the recognition accuracy is lower than the preset recognition accuracy threshold, the working mode of the image acquisition device can be configured to the first working mode (called free run mode). In the first working mode, the image acquisition frequency of the image acquisition device can be set to a first frequency (for example, 30fps or 60fps). In the first working mode, the perception base station will increase the detection frequency of the target image to improve the recognition rate of the identity information. This method becomes redundant detection. When it is determined that the identity information of the target object can be successfully identified, the working mode of the image acquisition device can be configured to the second working mode (called trigger mode). At this time, the image acquisition device can operate under the trigger signal of the perception base station, and the image acquisition frequency of the image acquisition device can be a second frequency lower than the first frequency (for example, 10fps).
[0099] The frame skipping step (i.e., the number of frames between two adjacent frames of the target image to be processed) for frame skipping processing of the target image in the second processing mode can be adjusted according to actual conditions, for example, according to the recognition accuracy rate. When the recognition accuracy rate is greater than the first recognition accuracy rate threshold, the frame skipping step is adjusted to the first step; when the recognition accuracy rate is greater than the second recognition accuracy rate threshold, the frame skipping step is adjusted to the second step. The first recognition accuracy rate threshold is greater than the second recognition accuracy rate threshold, and the first step is greater than the second step.
[0100] In the case where the target image includes a first target image and a second target image, the processing mode for processing the first target image may be different from the processing mode for processing the second target image. The first target image is acquired by a first image acquisition device (forward image acquisition device) in the perception base station, and the second target image is acquired by a second image acquisition device (backward image acquisition device) in the perception base station. Based on the processing mode for processing the first target image, the first target image may be processed to obtain a first processing result; based on the processing mode for processing the second target image, the second target image may be processed to obtain a second processing result; and then the perception result of the target object may be obtained according to the first processing result and the second processing result. The perception result may include the position and speed of the target object, and may also include feature information and identity information of the target object.
[0101] For example, if the time for starting the second image acquisition device is less than a preset time threshold, it can be determined that the processing mode for processing the second target image is the first processing mode. At the beginning of the activation of the second image acquisition device, by switching the processing mode to the first processing mode with a higher processing frame rate for processing the target object, it is convenient to obtain a more stable perception result, which is conducive to quickly resuming the tracking of the target object when the target object enters the untrackable area between the first image acquisition device and the second image acquisition device.
[0102] For another example, after determining that the processing mode for processing the second target image is the first processing mode, if the first processing result matches the second processing result, the processing mode for processing the second target image is switched from the first processing mode to the second processing mode.
[0103] In some embodiments, each image acquisition device may correspond to a target object list for recording each target object currently within the visual range of the image acquisition device. The target object list may include the number of target objects within the visual range of the image acquisition device, the identity information, feature information and location information of each target object, etc. When a new target object enters the visual range of the image acquisition device, and when the original target object leaves the visual range of the image acquisition device, the target object list will be refreshed. When the target object enters the untrackable area between the first image acquisition device and the second image acquisition device, the target object is deleted from the target object list of the first image acquisition device. If the deleted target object can be found in the target object list of the second image acquisition device, it means that the first processing result matches the second processing result. Optionally, the target object deleted from the target object list of the first image acquisition device can be preferentially perceived in the second target image.
[0104] If it is determined that the processing mode is the first processing mode, the target image is processed frame by frame; if it is determined that the processing mode is the second processing mode, the target image is skipped. In some embodiments, when the distance to the target object is less than a preset distance threshold, the target image is processed based on the processing mode. For example, when the target object just enters the perception area of the perception base station, the distance between the perception module (for example, the image acquisition device) in the perception base station and the target object may be far. At this time, even if the target image collected by the target object is processed, the processing effect may be poor, and a high recognition accuracy rate cannot be obtained. Therefore, the target image can be processed only when the target object travels to a position close to the perception module. In this way, the computing power resource usage of the system can be reduced.
[0105] Furthermore, the method of processing the target image includes at least any one of the following: tracking the target object in the target image and identifying the target object in the target image. Wherein, when the first condition is met, processing the target image includes tracking the target object in the target image and identifying the target object in the target image at the same time, and the first condition includes at least any one of the following: the recognition accuracy is lower than a preset recognition accuracy threshold; and the tracking of the target object fails, or the confidence of the identity information is lower than a preset confidence threshold. When the first condition is met, by simultaneously tracking and identifying the target object, it is possible to quickly re-establish the tracking of the target object when the tracking of the target object fails, and establish a relatively stable tracking result.
[0106] When the recognition accuracy is not lower than the recognition accuracy threshold, and the target object is tracked successfully, and the confidence of the acquired identity information of the target object is not lower than the confidence threshold, only the target object in the target image can be tracked without identifying the target object in the target image, so as to reduce processing complexity.
[0107] If the target image includes a first target image and a second target image, the way of processing the first target image may include at least any one of the following: tracking the target object in the first target image and identifying the target object in the first target image. The way of processing the second target image may include at least any one of the following: tracking the target object in the second target image and identifying the target object in the second target image.
[0108] Wherein, tracking or identifying the target object in the first target image may be identifying the target object in the first region of interest in the first target image. Similarly, tracking or identifying the target object in the second target image may be identifying the target object in the second region of interest in the second target image. Wherein, the region of interest may be a sub-region in the perception region, and the region of interest in each perception region may be predetermined. The region of interest is generally a sub-region in the perception region. For example, the perception region may include the target object driving region (e.g., a lane) and the region outside the target object driving region, while the region of interest only includes the target object driving region. In the case where the target object's driving region is a curve, the perception region may cover many areas outside the curve, but only the area inside the curve needs to be focused on, so the region of interest can be limited to the curve.
[0109] The method of processing the target image may be determined based on actual conditions, whether to track or identify the target object in the target image. For example, when the first condition is not met, processing the first target image includes tracking the target object in the first region of interest; the first condition is that the recognition accuracy of identifying the identity information of several previous target objects before the target object is greater than a preset recognition accuracy threshold. For another example, when the second condition is not met, processing the second target image includes tracking the target object in the second region of interest; the second condition is that the recognition accuracy of identifying the identity information of the target object is greater than a preset recognition accuracy threshold.
[0110] In the case of including a first image acquisition device and a second image acquisition device, the first target object acquired by the first image acquisition device and the second target image acquired by the second image acquisition device can be jointly optimized. When certain conditions are met, only the target objects in the first target image and / or the second target image are tracked without being recognized, thereby reducing complexity.
[0111] Furthermore, in actual application scenarios, when there are multiple perception base stations on a perception link, when a target object moves from the perception area of a previous perception base station to the perception area of a subsequent perception base station, the backward image acquisition device of the previous perception base station can be used as the forward image acquisition device of the subsequent perception base station, so as to perform joint processing of the target object by the two perception base stations.
[0112] The overall process of the target object perception process in some embodiments is as follows: Figure 6As shown. First, the target object list sent by the previous perception base station can be read, and the target object list can be used to determine whether there are target objects that need to be sensed within the perception area of this perception base station. If so, the forward image acquisition device (i.e., the front camera in the figure) is enabled to capture the first target image of the target object, and the recognition rate of the identity information (ID) of the target object is determined based on the first target image to determine whether it is greater than a preset recognition rate threshold. If it is greater, frame skipping tracking and frame skipping recognition are performed on the target object in the first target image captured by the forward image acquisition device. If it is not greater, the recognition rate of the identity information of the target object is continued to be determined based on the first target image until the recognition rate is greater than the recognition rate threshold. Based on the results of frame skipping tracking and frame skipping recognition, it is determined whether the target object has entered an untrackable area.
[0113] If so, notify the rearward image acquisition device (i.e., the rear camera in the figure) to capture a second target image of the target object. Track and identify the target object in the second target image captured by the rearward image acquisition device, and determine whether the identity information of the target object in the second target image is consistent with the identity information of the target object in the first target image based on the tracking and recognition results. If so, perform frame skipping tracking on the target object in the second target image. If the target object leaves the trackable area of the second image acquisition device, refresh the target object list and send the target object list to the next perception base station. Optionally, the recognition can be the recognition of the license plate number of the target object, or it can be the recognition of other characteristic information of the target object.
[0114] After processing the target image based on the processing mode to obtain the perception result of the target object, a map of the perception area can also be drawn according to the perception result. The perception module in the perception base station can use its own absolute geographic coordinates to provide real-time map drawing within the perception range of the perception base station. The drawn map may include at least any one of the following information: the number of lanes in the perception area, information about the lane where the target object is located, characteristic information of the target object on each lane, congestion information on each lane, and obstacle distribution information on each lane. Among them, the characteristic information of the target object may be information such as the color, vehicle model, speed, and lane of the target object. The congestion information of the lane may include indication information for indicating whether the lane is congested. Obstacles on the lane may include accumulated water, snow, etc. Figure 7 FIG. 1 is a schematic diagram of a map of some embodiments. The drawn map can be sent by the sensing base station to each target object in the sensing area, thereby providing assistance for the target object to perform automatic driving.
[0115] The perception result of the target object can be used for real-time non-sensing payment of the target object. A practical application scenario is as follows: there are multiple perception base stations arranged in a chain on the highway. The perception base stations are set along the road and can be set only within a distance in front of the pass service point (for example, a highway toll station). The perception base stations arranged in a chain constitute a perception link. Each perception base station covers a certain perception area, and each perception area overlaps. One or more vehicles (i.e., target objects) travel along the road section in the perception area until they reach the last perception area (i.e., the area covered by the perception base station at the toll station). During the driving process of the vehicle, the perception module (including the image acquisition device) in the perception base station where the vehicle is located is sequentially sensed and tracked, so that when the vehicle reaches the last perception area, the identity information of the vehicle can be recognized in real time and the highway toll can be paid from the account bound to the identity information of the vehicle. In this way, non-sensing payment can be realized when the vehicle passes through the highway toll station. It can be understood by those skilled in the art that the above application scenario embodiments are only used as exemplary illustrations and are not intended to limit the present disclosure.
[0116] like Figure 8 FIG. 2 is a target object perception method according to another embodiment of the present disclosure, the method comprising:
[0117] Step 802: Acquire a target image including a target object in a sensing area;
[0118] Step 804: determining a processing mode for processing the target image according to the current detection event; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image;
[0119] Step 806: Process the target image based on the processing mode to obtain a perception result of the target object.
[0120] The disclosed embodiment can determine the processing mode for processing the target image according to the current detection event, and different current detection events may correspond to different processing modes. In this way, different frame rates can be used to process the target image for different current detection events to meet the detection requirements of different detection events.
[0121] In some embodiments, the current detection event includes detecting the object information of the target object. Since different target objects may be suitable for different processing modes, by detecting the object information of the target object, different processing modes can be adopted for different target objects according to different object information. When the object information of the target object indicates that the target object is suitable for the first processing mode, the processing mode is switched to the first processing mode; when the object information of the target object indicates that the target object is suitable for the second processing mode, the processing mode is switched to the second processing mode. Taking the moving speed of the target object as an example, when the moving speed of the target object is greater than a preset speed threshold, the processing mode is determined to be the first processing mode, and the processing frame rate of the first processing mode is higher, so that the target object can be perceived more accurately and stably. When the moving speed of the target object is less than or equal to the speed threshold, the processing mode is determined to be the second processing mode, and the processing frame rate of the second processing mode is lower, so that processing resources can be saved and power consumption can be reduced.
[0122] Taking the type of the target object as an example, when the type of the target object is a preset type, the processing mode can be switched to the first processing mode; when the type of the target object is a type other than the preset type, the processing mode can be switched to the second processing mode. For example, when the target object is a vehicle, in the application scenario of detecting traffic violations on vehicles, since trucks account for a high proportion of traffic violation vehicles, the preset type can be a truck, and the first processing mode is used for trucks, and the second processing mode is used for vehicles other than trucks. For another example, when the public security department tracks and detects the vehicle of the criminal, it is known that the vehicle driven by the criminal is a blue seven-seater SUV, then the "blue seven-seater SUV" type can be set as the preset type, and the first processing mode is used for vehicles of the "blue seven-seater SUV" type, and the second processing mode is used for other types of vehicles.
[0123] In some other embodiments, the current detection event includes detecting a specific event. When a specific event is detected, the first processing mode can be used to improve the detection accuracy of the specific event. When the specific event is not detected, the second processing mode can be used to reduce power consumption and save processing resources. Taking the specific event as an example of a traffic violation event, when a traffic violation event is detected, the first processing mode can be used; when a traffic violation event is not detected, the second processing mode can be used. Wherein, the traffic violation event can be running a red light, speeding, the driver not wearing a seat belt, etc. Taking the specific event as an example of receiving a processing mode switching instruction, when a processing mode switching instruction to switch the processing mode to the first processing mode is received, the processing mode can be switched to the first processing mode; otherwise, the processing mode is determined to be the second processing mode. Wherein, the processing mode switching instruction can be sent by a user, and the user can be an operation and maintenance personnel of the perception base station. The user can send a processing mode switching instruction to the perception base station according to the actual situation to switch the processing mode to the first processing mode in a timely manner. In this way, the processing frame rate can be improved in some sudden situations.
[0124] In other embodiments, the current detection event may also include the current detection time and the current detection location. In the case where the current detection event includes the current detection time, if the current detection time is within the first time period, the processing mode may be determined to be the first processing mode, otherwise, the processing mode may be determined to be the second processing mode. Among them, the first time period may be preset, for example, 17:00 to 6:00 the next day may be set as the first time period. Since the light is weak in the above time period, it may affect the shooting effect of the target image. Therefore, by increasing the processing frame rate, the perception effect can be improved. The first time period can also be dynamically set according to the actual situation to meet the perception needs under different actual situations. In the case where the current detection event includes the current detection location, if the current detection location is a preset location, the processing mode may be determined to be the first processing mode, otherwise, the processing mode is determined to be the second processing mode. Among them, the preset location may be a place where traffic accidents frequently occur or a place with a poor shooting field of view. Increasing the processing frame rate at the above location can improve the perception effect.
[0125] Other embodiments of the disclosed embodiments are similar to the aforementioned embodiments of the target object perception method and will not be described in detail here.
[0126] like Fig. 9 As shown, the embodiment of the present disclosure also provides a target object perception method, the method comprising:
[0127] Step 902: Determine the current detection event;
[0128] Step 904: Determine a shooting mode for shooting the target object in the perception area according to the current detection event; wherein the shooting mode includes a first shooting mode for shooting the target object at a first frame rate, and a second shooting mode for shooting the target object at a second frame rate, and the first frame rate is greater than the second frame rate.
[0129] The disclosed embodiment controls the shooting mode of the target image of the target object captured by the image acquisition device. In certain cases, the shooting mode of the image acquisition device is controlled to be a first shooting mode with a higher shooting frame rate. In other cases, the shooting mode of the image acquisition device is controlled to be a second shooting mode with a lower shooting frame rate. The captured target image can be used to obtain the perception result of the target object. In certain cases, redundant detection of the target object is achieved by adopting the first shooting mode, so that more accurate perception results can be obtained later. In non-specific cases, the second shooting mode is adopted to reduce system power consumption and reduce data processing volume.
[0130] In some embodiments, the current detection event may include detecting object information of the target object. The object information includes at least any one of the following: the moving speed and type of the target object. In the case where the object information includes the moving speed of the target object, if the moving speed is greater than a preset speed threshold, the shooting mode is determined to be the first shooting mode; if the moving speed is not greater than the speed threshold, the shooting mode is determined to be the second shooting mode. In the case where the object information includes the type of the target object, if the type is a preset type, the shooting mode is determined to be the first shooting mode; if the type is not a preset type, the shooting mode is determined to be the second shooting mode.
[0131] In some embodiments, the current detection event may include detecting a specific event. The specific event includes a traffic violation event and / or receiving a shooting mode switching instruction. When a traffic violation event is detected and / or a shooting mode switching instruction to switch the shooting mode to the first shooting mode is received, the shooting mode is determined to be the first shooting mode; otherwise, the shooting mode is determined to be the second shooting mode.
[0132] Other embodiments of the disclosed embodiments are similar to the aforementioned embodiments of the target object perception method and will not be described in detail here.
[0133] like Fig.10 As shown, the embodiment of the present disclosure also provides a target object perception method, the method comprising:
[0134] Step 1002: Acquire multiple frames of target images including the target object in the sensing area;
[0135] Step 1004: Determine a sending mode for sending the multiple frames of target images to a processing unit for processing based on a current detection event; wherein the sending mode includes a first sending mode for sending the multiple frames of target images frame by frame, and a second sending mode for sending the multiple frames of target images by skipping frames.
[0136] In an embodiment of the present disclosure, a multi-frame target image including a target object can be acquired by an image acquisition device, and then all or part of the target images can be selectively sent to a processing unit of a perception base station for processing to obtain a perception result of the target object. In certain cases, the sending mode of the image acquisition device is controlled to be a first sending mode with a higher sending frame rate, and in other cases, the sending mode of the image acquisition device is controlled to be a second sending mode with a lower sending frame rate. The sent target image can be used by the processing unit to obtain the perception result of the target object. In certain cases, redundant detection of the target object is achieved by adopting the first sending mode, so that more accurate perception results can be obtained later. In non-specific cases, the second sending mode is adopted to reduce system power consumption and reduce data processing volume.
[0137] In some embodiments, the current detection event may include detecting object information of the target object. The object information includes at least any one of the following: the moving speed and type of the target object. In the case where the object information includes the moving speed of the target object, if the moving speed is greater than a preset speed threshold, the sending mode is determined to be the first sending mode; if the moving speed is not greater than the speed threshold, the sending mode is determined to be the second sending mode. In the case where the object information includes the type of the target object, if the type is a preset type, the sending mode is determined to be the first sending mode; if the type is not a preset type, the sending mode is determined to be the second sending mode.
[0138] In some embodiments, the current detection event may include detecting a specific event. The specific event includes a traffic violation event and / or receiving a transmission mode switching instruction. In the case where a traffic violation event is detected and / or a transmission mode switching instruction to switch the transmission mode to the first transmission mode is received, the transmission mode is determined to be the first transmission mode; otherwise, the transmission mode is determined to be the second transmission mode.
[0139] In the second sending mode, the frame skipping step of the frame skipping sending can be adjusted dynamically. For example, the frame skipping step can be adjusted to 1, that is, the target image is sent once every 1 frame (for example, the image frames such as the 1st frame, the 3rd frame, the 5th frame, etc. in the target image are sent to the processing unit). For another example, the frame skipping step can be adjusted to 3, that is, the target image is sent once every 3 frames (for example, the image frames such as the 1st frame, the 5th frame, the 9th frame, etc. in the target image are sent to the processing unit). The frame skipping step can also be other values, and the specific value can be determined according to the current detection event. For example, the current detection event is to detect the moving speed of the target object. When the moving speed is greater than or equal to the first speed threshold, the frame skipping step can be adjusted to the first step; when the moving speed is greater than or equal to the second speed threshold and less than the first speed threshold, the frame skipping step is adjusted to the second step, wherein the first step is less than the second step, and the first speed threshold is greater than the second speed threshold. The division interval of the frame skipping step can also be greater than two intervals, and the specific division method depends on the actual situation.
[0140] Other embodiments of the disclosed embodiments are similar to the aforementioned embodiments of the target object perception method and will not be described in detail here.
[0141] like Fig.11 As shown, the embodiment of the present disclosure also provides a target object perception method, the method comprising:
[0142] Step 1102: acquiring target images of multiple target objects in the sensing area respectively;
[0143] Step 1104: Use different processing modes to process the target image of each target object, wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames on the target image. The processing mode for processing the target image of each target object is determined according to a current detection event related to the target object.
[0144] In the embodiment of the present disclosure, the same perception area may include multiple target objects at the same time, so the target images corresponding to each target object can be obtained separately. For example, N1 frames of target images including target object 1 can be obtained, N2 frames of target images including target object 2 can be obtained, and so on. Different processing modes can be used for processing the target images of each target object. For example, N1 frames of target images including target object 1 can be processed using a first processing mode, and N2 frames of target images including target object 2 can be processed using a second processing mode. The processing modes corresponding to the target images of different target objects may be the same or different. By distinguishing different processing modes for different target objects, on the one hand, for target objects that need to adopt the first processing mode, the processing effect of their target images is guaranteed, thereby improving the perception accuracy of these target objects; on the other hand, for target objects that only need to adopt the second processing mode, the processing resource consumption of the target images of these target objects is reduced, thereby reducing system power consumption.
[0145] In some embodiments, the current detection event may include detecting object information of the target object. Wherein, the object information includes at least any of the following: the moving speed and type of the target object. In the case where the object information includes the moving speed of the target object, if the moving speed of the target object is greater than a preset speed threshold, the processing mode of the target image of the target object is determined to be the first processing mode; if the moving speed of the target object is not greater than the speed threshold, the processing mode of the target image of the target object is determined to be the second processing mode. In the case where the object information includes the type of the target object, if the type of the target object is a preset type, the processing mode of the target image of the target object is determined to be the first processing mode; if the type of the target object is not a preset type, the processing mode of the target image of the target object is determined to be the second processing mode.
[0146] In some embodiments, the current detection event may include detecting a specific event. The specific event includes a traffic violation event and / or receiving a processing mode switching instruction. When a traffic violation of the target object is detected and / or a processing mode switching instruction is received to switch the processing mode of the target image of the target object to the first processing mode, the processing mode is determined to be the first processing mode; otherwise, the processing mode is determined to be the second processing mode.
[0147] Other embodiments of the disclosed embodiments are similar to the aforementioned embodiments of the target object perception method and will not be described in detail here.
[0148] like Fig.12As shown, it is an overall flow chart of image acquisition, image transmission and image processing of the embodiment of the present disclosure. In the embodiment of the present disclosure, the shooting mode of the image acquisition device is first determined. If the shooting mode is determined to be the first shooting mode, the target image of the target object is acquired at the first frame rate; if the shooting mode is determined to be the second shooting mode, the target image of the target object is acquired at the second frame rate. The acquired target image is sent to the processing unit by the image acquisition device. Therefore, it is necessary to determine the sending mode. If the sending mode is determined to be the first sending mode, the target image is sent to the processing unit frame by frame; if the sending mode is determined to be the second sending mode, the target image is sent to the processing unit by skipping frames. The frame skipping step of the frame skipping transmission can be adjusted dynamically. After receiving the target image, the processing unit can process the target image. First, the processing mode is determined. If the processing mode is determined to be the first processing mode, the received target image is processed frame by frame; if the processing mode is determined to be the second processing mode, the received target image is subjected to frame skipping processing. The frame skipping step of the frame skipping processing can be adjusted dynamically. Then, the target image is processed according to the determined processing mode to obtain the perception result of the target object. If there are multiple target objects, the above process can be executed for each target object respectively.
[0149] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0150] like Fig.13 As shown, the present disclosure also provides a target object perception device, the device comprising:
[0151] A first acquisition module 1302 is used to acquire a target image including a target object in a sensing area;
[0152] The first determination module 1304 is used to determine the recognition accuracy of the identity information of the target object, and determine the processing mode for processing the target image according to the recognition accuracy; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image;
[0153] The first perception module 1306 is used to process the target image based on the processing mode to obtain a perception result of the target object.
[0154] like Fig.14 As shown, the present disclosure also provides a target object perception device, the device comprising:
[0155] The second acquisition module 1402 is used to acquire a target image including a target object in the sensing area;
[0156] A second determination module 1404 is used to determine a processing mode for processing the target image according to the current detection event; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image;
[0157] The second perception module 1406 is used to process the target image based on the processing mode to obtain a perception result of the target object.
[0158] like Fig.15 As shown, the present disclosure also provides a target object perception device, the device comprising:
[0159] The third determination module 1502 is used to determine the current detection event;
[0160] The fourth determination module 1504 is used to determine a shooting mode for shooting the target object in the perception area according to the current detection event; wherein the shooting mode includes a first shooting mode for shooting the target object at a first frame rate, and a second shooting mode for shooting the target object at a second frame rate, and the first frame rate is greater than the second frame rate.
[0161] like Fig.16 As shown, the present disclosure also provides a target object perception device, the device comprising:
[0162] The third acquisition module 1602 is used to acquire multiple frames of target images including the target object in the sensing area;
[0163] The fifth determination module 1604 is used to determine a sending mode for sending the multiple frames of target images to a processing unit for processing according to a current detection event; wherein the sending mode includes a first sending mode for sending the multiple frames of target images frame by frame, and a second sending mode for sending the multiple frames of target images by skipping frames.
[0164] like Fig.17 As shown, the present disclosure also provides a target object perception device, the device comprising:
[0165] The fourth acquisition module 1702 is used to respectively acquire target images of multiple target objects in the sensing area;
[0166] The third perception module 1704 is used to process the target image of each target object using different processing modes to obtain the perception results of each target object respectively;
[0167] Among them, the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image. The processing mode for processing the target image of each target object is determined according to the current detection event related to the target object.
[0168] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0169] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.
[0170] An embodiment of the present disclosure further provides a perception base station, which comprises at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in any of the above embodiments is implemented when the processor executes the program.
[0171] Fig.18 A more specific hardware structure diagram of a perception base station provided by an embodiment of the present disclosure is shown, and the device may include: a processor 1802, a memory 1804, an input / output interface 1806, a communication interface 1808, and a bus 1810. The processor 1802, the memory 1804, the input / output interface 1806, and the communication interface 1808 are connected to each other in communication within the device through the bus 1810.
[0172] The processor 1802 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present disclosure.
[0173] The memory 1804 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1804 may store an operating system and other application programs. When the technical solution provided by the embodiment of the present disclosure is implemented by software or firmware, the relevant program code is stored in the memory 1804 and is called and executed by the processor 1802.
[0174] The input / output interface 1806 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0175] The communication interface 1808 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0176] The bus 1810 comprises a pathway for transmitting information between the various components of the device (eg, the processor 1802, the memory 1804, the input / output interface 1806, and the communication interface 1808).
[0177] It should be noted that, although the above device only shows the processor 1802, the memory 1804, the input / output interface 1806, the communication interface 1808 and the bus 1810, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.
[0178] like Fig.19 As shown, an embodiment of the present disclosure further provides a perception system, and the perception system includes the perception base station 1902 described in any of the above embodiments.
[0179] In some embodiments, the perception system further includes: an image acquisition device 1904; the image acquisition device 1904 is used to acquire the target image. The image acquisition device 1904 can be installed on the perception base station 1902.
[0180] In some embodiments, the image acquisition device 1904 includes: a first image acquisition device 1904a and a second image acquisition device 1904b; the first image acquisition device 1904a is used to obtain a first target image including the target object when the target object enters the perception area, and the second image acquisition device 1904b is used to obtain a second target image including the target object when the target object exits the perception area.
[0181] The embodiments of the present disclosure further provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment is implemented.
[0182] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0183] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure or some parts of the embodiments.
[0184] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.
[0185] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor.
[0186] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the embodiment of the present disclosure.
Claims
1. A target object perception method, the method comprising: Acquire a target image including a target object in a sensing area; Determine a recognition accuracy rate for identifying the identity information of the target object, and determine a processing mode for processing the target image according to the recognition accuracy rate; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image; wherein, if the recognition accuracy rate is lower than a preset recognition accuracy rate threshold, determine that the processing mode is the first processing mode; and / or if the recognition accuracy rate is not lower than the recognition accuracy rate threshold, determine that the processing mode is the second processing mode; The target image is processed based on the processing mode to obtain a perception result of the target object.
2. The method according to claim 1, wherein the method is performed by a first sensing base station in a sensing link, wherein the sensing link includes a plurality of sensing base stations including the first sensing base station; The method further comprises: Acquire first characteristic information of the target object sent by a previous perception base station of the first perception base station and second characteristic information of the target object in the first perception base station; The identity information of the target object is identified according to the first feature information and the second feature information.
3. The method according to claim 1, further comprising: Obtaining the original image of the sensing area; A target image including the target object is screened out from the original image.
4. The method according to claim 3, wherein the step of filtering out a target image including the target object from the original image comprises: The original image is detected by a deep learning algorithm to detect a target image including the target object from the original image.
5. The method according to claim 1, wherein processing the target image based on the processing mode comprises: When the distance between the target object and the target object is less than a preset distance threshold, the target image is processed based on the processing mode.
6. The method according to claim 1, further comprising: After the target image is processed based on the processing mode to obtain a perception result of the target object, a map of the perception area is drawn according to the perception result.
7. According to the method of claim 6, the map includes at least any one of the following: the number of lanes in the perception area, information about the lane where the target object is located, feature information of the target object on each lane, congestion information on each lane, and obstacle distribution information on each lane.
8. The method according to claim 1, wherein the target image comprises a first target image acquired when the target object enters the sensing area, and a second target image acquired when the target object exits the sensing area; the processing mode comprises a processing mode for processing the first target image and a processing mode for processing the second target image; The processing of the target image based on the processing mode to obtain a perception result of the target object includes: Based on the processing mode for processing the first target image, the first target image is processed to obtain a first processing result; Based on the processing mode for processing the second target image, processing the second target image to obtain a second processing result; The perception result of the target object is obtained according to the first processing result and the second processing result.
9. The method according to claim 8, wherein the first target image is acquired by a first image acquisition device, and the second target image is acquired by a second image acquisition device.
10. The method according to claim 9, wherein acquiring a target image including a target object in a sensing area comprises: When the target object enters a non-trackable area between the first image acquisition device and the second image acquisition device, the second image acquisition device is enabled to acquire the second target image.
11. The method according to claim 9, further comprising: If the time for starting the second image acquisition device is less than a preset time threshold, it is determined that the processing mode for processing the second target image is the first processing mode.
12. The method according to claim 11, further comprising: After determining that the processing mode for processing the second target image is the first processing mode, if the first processing result matches the second processing result, the processing mode for processing the second target image is switched from the first processing mode to the second processing mode. 13 . The method according to claim 9 , wherein the second image acquisition device is disabled if the first image acquisition device does not acquire the first target image including the target object.
14. The method according to claim 9, further comprising: If the first image acquisition device has not acquired the first target image including the target object at the current moment, and the first image acquisition device has acquired the first target image including the target object within a period of time before the current moment, the second image acquisition device is enabled to obtain the second target image.
15. The method according to claim 9, wherein processing the first target image comprises: Identifying a target object in a first region of interest in the first target image; and / or Tracking a target object in a first region of interest in the first target image; and / or The processing of the second target image comprises: identifying a target object in a second region of interest in the second target image; and / or Tracking a target object in a second region of interest in the second target image.
16. The method according to claim 15, wherein, when a first condition is not satisfied, the processing of the first target image comprises tracking the target object in the first region of interest; the first condition being that a recognition accuracy rate of identifying identity information of a plurality of previous target objects before the target object is greater than a preset recognition accuracy rate threshold; and / or When the second condition is not met, the processing of the second target image includes tracking the target object in the second area of interest; the second condition is: the recognition accuracy of the identity information of the target object is greater than a preset recognition accuracy threshold.
17. The method according to claim 1, wherein the target object is added to the target object list when entering the perception area; The method further comprises: In response to the target object leaving the trackable area, the target object list is refreshed.
18. The method according to claim 1, wherein, when a first condition is satisfied, processing the target image comprises tracking a target object in the target image and identifying the target object in the target image; the first condition comprises at least any one of the following: The recognition accuracy is lower than a preset recognition accuracy threshold; and Tracking of the target object fails, or the confidence level of the identity information is lower than a preset confidence level threshold.
19. The method according to claim 1, further comprising: If the identity information of the target object is not detected, the identity information is reallocated to the target object.
20. A target object perception method, the method comprising: Acquire a target image including a target object in a sensing area; Determine a processing mode for processing the target image according to the current detection event; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image; Processing the target image based on the processing mode to obtain a perception result of the target object; The current detection event includes detecting a specific event, and the specific event includes a traffic violation event; and determining a processing mode for processing the target image according to the current detection event includes: If the fourth condition is met, the processing mode is determined to be the first processing mode; and / or if the fourth condition is not met, the processing mode is determined to be the second processing mode; wherein the fourth condition includes: the violation event is detected. The method according to claim 20 , wherein the current detection event further comprises detecting object information of the target object.
22. The method according to claim 21, wherein the object information comprises at least any one of the following: a moving speed and a type of the target object.
23. The method according to claim 22, wherein determining a processing mode for processing the target image according to a current detection event comprises: If the object information satisfies a third condition, determining that the processing mode is the first processing mode; and / or if the object information does not satisfy the third condition, determining the processing mode to be the second processing mode; The third condition includes: the moving speed of the target object is greater than a preset speed threshold, and / or the type of the target object is a preset type.
24. The method according to claim 21, wherein the specific event further comprises receiving a processing mode switching instruction.
25. The method according to claim 24, wherein determining a processing mode for processing the target image according to a current detection event comprises: If a processing mode switching instruction for switching the processing mode to the first processing mode is received, determining that the processing mode is the first processing mode; And / or if no processing mode switching instruction to switch the processing mode to the first processing mode is received, determining that the processing mode is the second processing mode.
26. A target object perception method, the method comprising: respectively acquiring target images of multiple target objects in the sensing area; Using different processing modes to process the target image of each target object, wherein the processing mode includes a first processing mode for processing the target image frame by frame and a second processing mode for performing frame skipping processing on the target image, and the processing mode for processing the target image of each target object is determined according to a current detection event related to the target object; The current detection event includes detecting a specific event, and the specific event includes a traffic violation event. When a traffic violation is detected for a target object, a processing mode of a target image of the target object is determined to be the first processing mode; otherwise, a processing mode of a target image of the target object is determined to be the second processing mode.
27. A target object sensing device, the device comprising: A first acquisition module is used to acquire a target image including a target object in a sensing area; A first determination module is used to determine the recognition accuracy of identifying the identity information of the target object, and determine a processing mode for processing the target image according to the recognition accuracy; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for skipping frames of the target image; wherein, if the recognition accuracy is lower than a preset recognition accuracy threshold, the processing mode is determined to be the first processing mode; and / or if the recognition accuracy is not lower than the recognition accuracy threshold, the processing mode is determined to be the second processing mode; The first perception module is used to process the target image based on the processing mode to obtain a perception result of the target object.
28. A target object sensing device, the device comprising: A second acquisition module is used to acquire a target image including a target object in the perception area; A second determination module is used to determine a processing mode for processing the target image according to a current detection event; wherein the processing mode includes a first processing mode for processing the target image frame by frame, and a second processing mode for performing frame skipping processing on the target image; A second perception module, used to process the target image based on the processing mode to obtain a perception result of the target object; The current detection event includes detecting a specific event, and the specific event includes a traffic violation event; and determining a processing mode for processing the target image according to the current detection event includes: If the fourth condition is met, the processing mode is determined to be the first processing mode; and / or if the fourth condition is not met, the processing mode is determined to be the second processing mode; wherein the fourth condition includes: the violation event is detected.
29. A target object sensing device, the device comprising: A fourth acquisition module is used to respectively acquire target images of multiple target objects in the sensing area; A third perception module is used to process the target image of each target object using different processing modes to obtain the perception results of each target object respectively; The processing modes include a first processing mode for processing the target image frame by frame, and a second processing mode for processing the target image frame by frame, and the processing mode for processing the target image of each target object is determined according to the current detection event related to the target object; The current detection event includes detecting a specific event, and the specific event includes a traffic violation event. When a traffic violation is detected for a target object, a processing mode of a target image of the target object is determined to be the first processing mode; otherwise, a processing mode of a target image of the target object is determined to be the second processing mode.
30. A perception base station, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 26 when executing the program.
31. A perception system, comprising the perception base station according to claim 30.
32. The sensing system according to claim 31, further comprising: Image acquisition device; The image acquisition device is used to acquire the target image.
33. The perception system according to claim 32, wherein the image acquisition device comprises: a first image acquisition device and a second image acquisition device; The first image acquisition device is used to acquire a first target image including the target object when the target object enters the perception area, and the second image acquisition device is used to acquire a second target image including the target object when the target object exits the perception area.
34. A computer-readable storage medium, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 26 when executing the program.
