Detection box acquisition method and related equipment

Abstract

The invention provides a detection frame acquisition method and related equipment, and the method comprises the steps: obtaining continuous frames of environment images in a vehicle driving process; performing obstacle recognition on each frame of environment image to obtain detection frame information of obstacles in each frame of environment image; generating a first detection frame sequence of the current frame according to the acquisition time of the detection frame information and the detection frame information; generating prediction detection frame information of the current frame according to the detection frame information of the previous frame, and generating a second detection frame sequence of the current frame according to the prediction detection frame information of the current frame; and matching the first detection frame sequence and the second detection frame sequence of the same frame, and determining a target detection frame of the obstacle according to a matching result. According to the invention, accurate detection of the object can be realized, and the accuracy of obtaining the information of the detection frame is improved.

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a detection frame acquisition method and related equipment. Background Art

[0002] In recent years, with the rapid growth in the number of traffic vehicles, traffic conditions have become more complicated. When monitoring actual road scenes, it is often necessary to identify road scenes through detection frames. In the process of identifying detection frames, the existing technology often identifies multiple detection frames. When removing duplicate detection frames, the existing recognition algorithm is prone to overfitting of detection frames, resulting in accidental deletion of detection frames. For example, when identifying a vehicle, one vehicle is identified as multiple vehicles, such as identifying the front and rear of the same vehicle as one vehicle; or identifying multiple vehicles as one vehicle, resulting in the vehicle obtaining the wrong road scene, affecting the vehicle's driving judgment.

[0003] How to provide a more accurate detection frame acquisition method for vehicles is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present application provides a detection frame acquisition method and related equipment, which can achieve accurate detection of objects and improve the accuracy of acquiring detection frame information.

[0005] A first aspect of an embodiment of the present application provides a detection frame acquisition method, including: acquiring continuous frames of environmental images during vehicle driving; performing obstacle recognition on each frame of the environmental image to obtain detection frame information of the obstacle in each frame of the environmental image; generating a first detection frame sequence of the current frame based on the acquisition time of the detection frame information and the detection frame information; generating predicted detection frame information of the current frame based on the detection frame information of the previous frame, and generating a second detection frame sequence of the current frame based on the predicted detection frame information of the current frame; matching the first detection frame sequence and the second detection frame sequence of the same frame, and determining a target detection frame of the obstacle based on the matching result.

[0006] Compared with the related art, the embodiments of the present application have at least the following advantages: This application obtains the first detection frame sequence corresponding to the obstacle through the environmental images of continuous frames to realize the detection frame trajectory tracking of the same obstacle in continuous frames. At the same time, this application constructs a second detection frame sequence of the obstacle by predicting the trajectory information of the current frame, and the second detection frame sequence is used to further track the trajectory information of the obstacle. By matching the first detection frame sequence and the second detection frame sequence of the same frame, the detection frame trajectory can be verified while tracking the detection frame trajectory. According to the matching results, it can be determined whether the obstacle has moved or disappeared, thereby avoiding the acquisition of the detection frame information of a fixed object and improving the accuracy of the detection frame information.

[0007] Optionally, obstacle recognition is performed on each frame of the environmental image to obtain detection frame information of the obstacles in each frame of the environmental image, including: based on a detection frame detection algorithm, obstacle recognition is performed on each frame of the environmental image to obtain a first-category detection frame corresponding to the obstacle and the confidence of the first-category detection frame; based on the confidence of the first-category detection frame, the credibility of the first-category detection frame is calculated; the credibility threshold of the first-category detection frame is obtained, and the first-category detection frame that does not meet the credibility threshold is eliminated to obtain a second-category detection frame; the positioning signal corresponding to the environmental image is obtained, and based on the second-category detection frame and the positioning signal, the coordinates of the second-category detection frame are converted into the world coordinate system to obtain a third-category detection frame; and the detection frame information is generated according to the third-category detection frame and the confidence of the third-category detection frame.

[0008] Optionally, predicted detection frame information of the current frame is generated based on the detection frame information of the previous frame, and a second detection frame sequence of the current frame is generated based on the predicted detection frame information of the current frame, including: at an initial moment, the second detection frame sequence is the same as the first detection frame sequence; after the initial moment, the detection frame information of the previous frame is obtained, and the predicted detection frame information of the next frame is generated based on the detection frame information of the previous frame, and a second detection frame sequence is generated based on the first detection frame sequence of the previous frame and the predicted detection frame information of the next frame.

[0009] Optionally, the first detection frame sequence and the second detection frame sequence of the same frame are matched, and the target detection frame of the obstacle is determined according to the matching result, including: obtaining the detection frame information and the predicted detection frame information of the same frame according to the first detection frame sequence and the second detection frame sequence; matching the detection frame information and the predicted detection frame information of the same frame to obtain a matching result; wherein the matching result includes: a first matching result, a second matching result and a third matching result, wherein the first matching result is that the second detection frame sequence does not receive the detection frame information corresponding to the current frame; the second matching result is that the detection frame information and the predicted detection frame information of the same frame do not match; the third matching result is that the detection frame information and the predicted detection frame information of the same frame match; the second detection frame sequence is updated according to the matching result to obtain the third detection frame sequence of the obstacle; and the target detection frame of the obstacle is determined according to the third detection frame sequence corresponding to all obstacles in the environment image.

[0010] Optionally, updating the second detection frame sequence according to the matching result includes: when the matching result is the first matching result; obtaining the confidence of the second detection frame sequence of the current frame; obtaining a preset confidence difference, and reducing the confidence of the second detection frame sequence according to the preset confidence difference; obtaining the confidence of the updated second detection frame sequence, and when the confidence of the updated second detection frame sequence is less than a preset threshold, deleting the second detection frame sequence corresponding to the obstacle.

[0011] Optionally updating the second detection frame sequence according to the matching result, including: when the matching result is the second matching result; obtaining unmatched detection frame information, dividing the unmatched detection frame information into a first detection frame sequence of the newly added obstacle; generating a second detection frame sequence of the newly added obstacle according to the first detection frame sequence of the newly added obstacle.

[0012] Optionally, updating the second detection frame sequence according to the matching result includes: when the matching result is the third matching result; obtaining a first confidence of the first detection frame sequence and a second confidence of the second detection frame sequence; calculating an updated confidence based on the first confidence and the second confidence; obtaining the detection frame information of the current frame, replacing the predicted detection frame information of the current frame in the second detection frame sequence with the detection frame information of the current frame, and updating the second detection frame sequence.

[0013] In the second aspect, an embodiment of the present application also provides a detection frame acquisition device, including: an acquisition unit, used to acquire continuous frames of environmental images during vehicle driving; an identification unit, used to perform obstacle identification on each frame of the environmental image, and obtain detection frame information of the obstacles in each frame of the environmental image; a detection unit, used to generate a first detection frame sequence of the current frame according to the acquisition time of the detection frame information and the detection frame information; a prediction unit, used to generate predicted detection frame information of the current frame according to the detection frame information of the previous frame, and generate a second detection frame sequence of the current frame according to the predicted detection frame information of the current frame; a result unit, used to match the first detection frame sequence and the second detection frame sequence of the same frame, and determine the target detection frame of the obstacle according to the matching result.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the detection frame acquisition method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the detection frame acquisition method described in the first aspect.

[0016] The technical effects obtained by the above-mentioned second, third and fourth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of the steps of a method for obtaining a detection frame provided in one embodiment of the present application.

[0018] Figure 2 A flowchart of the steps of a detection frame acquisition method provided in yet another embodiment of the present application.

[0019] Figure 3 A functional module diagram of a detection frame acquisition device provided in one embodiment of the present application.

[0020] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the implementation methods of the present application and the features in the implementation methods can be combined with each other without conflict.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present application. The described implementations are only part of the implementations of the present application, rather than all the implementations.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0024] It should be further noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0025] In this application, "at least one" means one or more, and "more" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0026] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0027] In recent years, with the rapid growth in the number of traffic vehicles, traffic conditions have become more complicated. When monitoring actual road scenes, it is often necessary to identify road scenes through detection frames. In the process of identifying detection frames, the existing technology often identifies multiple detection frames. When removing duplicate detection frames, the existing recognition algorithm is prone to overfitting of detection frames, resulting in accidental deletion of detection frames. For example, when identifying a vehicle, one vehicle is identified as multiple vehicles, such as identifying the front and rear of the same vehicle as one vehicle; or identifying multiple vehicles as one vehicle, resulting in the vehicle obtaining the wrong road scene, affecting the vehicle's driving judgment.

[0028] How to provide a more accurate detection frame acquisition method for vehicles is an urgent problem to be solved.

[0029] In view of the above, the embodiments of the present application provide a detection frame acquisition method and related devices, which can realize data analysis of specific services and improve the stability of data analysis.

[0030] The specific process of this embodiment is as follows Figure 1 As shown, the following steps are included: Step 101, while the vehicle is traveling, continuous frames of environmental images are acquired.

[0031] The continuous frames of the environment image may be from a road video captured by a vehicle camera. The obstacles may be all movable objects included in the environment image. As an example, the obstacles may be vehicles, pedestrians, etc. included in the environment image.

[0032] Step 102: perform obstacle recognition on each frame of the environment image to obtain detection frame information of the obstacle in each frame of the environment image.

[0033] The detection frame information includes but is not limited to the detection frame, the generation event of the detection frame and the generation location of the detection frame. The detection frame is used to characterize the area corresponding to the obstacle in the environment image. Each obstacle corresponds to a detection frame, and the detection frame information of all obstacles in the environment image constitutes the detection frame set corresponding to the environment image. The same obstacle in the environment image of consecutive frames is tracked to generate the detection frame sequence of the same obstacle. The detection frame sequences of all obstacles constitute the detection frame sequence set corresponding to the environment image of the consecutive frames.

[0034] As an example, the execution subject can determine the detection frame of the obstacle in the environment image through the detection frame recognition model. The detection frame recognition model is used to characterize the corresponding relationship between the environment image and the detection frame of the obstacle. The detection frame recognition model can be trained based on a machine learning algorithm using a network model such as a convolutional neural network, a recurrent neural network, and a residual neural network.

[0035] In the related art, when the environment image is input into the model detection, the model is decoded to obtain the detection frame, and then the NMS (Non-Maximum Suppression) algorithm is used to deduplicate all the generated detection frames, and finally the target detection frame is output. The NMS method is a widely used technology in target detection, which is used to merge candidate boxes (Bounding Boxes) to ensure that each object is detected only once. However, the structure of the existing NMS algorithm is not accurate. This application provides a more accurate method to replace the NMS algorithm to deduplicate the model decoded detection frame.

[0036] In some optional implementations of this embodiment, for each frame of the environment image, the execution subject can determine the detection frame information of the obstacle based on a preset detection algorithm to obtain a detection frame information set. The present application filters and corrects the preset detection algorithm.

[0037] In one embodiment, if Figure 2 As shown, obstacle recognition is performed on each frame of the environment image to obtain the detection frame information of the obstacle in each frame of the environment image, including: Step 201 : Based on the detection frame detection algorithm, obstacle recognition is performed on each frame of the environment image to obtain the first type of detection frame corresponding to the obstacle and the confidence of the first type of detection frame.

[0038] Step 202: Calculate the credibility of the first type of detection box based on the confidence of the first type of detection box.

[0039] Step 203, obtaining a credibility threshold of the first category detection frame, removing the first category detection frame that does not meet the credibility threshold, and obtaining the second category detection frame.

[0040] Step 204, obtaining a positioning signal corresponding to the environment image, and based on the second type of detection frame and the positioning signal, converting the coordinates of the second type of detection frame into a world coordinate system to obtain a third type of detection frame.

[0041] Step 205: Generate detection frame information according to the third category detection frame and the confidence of the third category detection frame.

[0042] In step 201, input signals are identified, which are environment images and related information of the environment images, detection frames output by the model, and vehicle global positioning signals, such as wheel speed recursive signals, RTK, and the like.

[0043] In steps 202 and 203, to remove highly overlapping detection boxes, the detection box detection algorithm is prone to generating multiple detection boxes near the same obstacle, and the positions of each detection box are basically the same. The distance between the detection boxes of the same obstacle can be directly calculated. For example, calculate the distance between the first detection box and the second detection box, and take this distance as the confidence level, denoted as d. Select the distance threshold D as the confidence threshold. When d < D, delete the detection box with the lower confidence level among the two boxes. It should be noted that the value of the distance threshold D must be very small. If the selected distance threshold is relatively large, the algorithm will not achieve the desired effect. In this embodiment, the highly repetitive detection boxes are removed according to the confidence level to simplify the computational complexity of subsequent obtaining the target detection boxes.

[0044] In step 204, according to the positioning signal, the second type of detection box in the ego-vehicle coordinate system is converted into the third type of detection box in the world coordinate system. Set a 3×3 rotation matrix R and a 1×3 translation matrix T. According to the rotation matrix R, the yaw angle of rotation around the Z axis can be obtained. Set x, y, z, and sita as variables in the ego-vehicle coordinate system to obtain the second type of detection box boxA(x, y, z, l, w, h, sita) in the ego-vehicle coordinate system. For the third type of detection box of the detection box boxA in the global coordinate system after conversion, the third type of detection box is boxB(x1, y1, z1, l, w, h, sita1), and give the third type of detection box to the first detection box sequence detset. Among them, the conversion formulas of the second type of detection box and the third type of detection box are shown in the following formulas (1) and (2), including: Formula (1); Formula (2); Step 103, generate the first detection box sequence of the current frame according to the acquisition time and information of the detection box.

[0045] For the first detection frame sequence of the same obstacle in consecutive frames, Detbox{box1_1, box1_2, box1_3} is used to represent it, where box1 in box1_X indicates that the detection target is the first detection target, and X indicates the detection frame recognized by the environmental image of the Xth frame. Each first detection frame sequence corresponds to a confidence level, and the confidence level corresponding to the first detection frame sequence is the confidence level of the detection frame of the current frame. For the environmental image of the same frame, the environmental image of the same frame includes: multiple obstacles, each obstacle corresponds to a Detbox, for example, Detbox1 is used to represent the first obstacle, Detbox2 is used to represent the second obstacle, and Detboxi is used to represent the i-th obstacle. For all obstacles, the set of the first detection frame sequence Detset{Detbox1, Detbox2...} is obtained. It should be noted that in each detection frame sequence, the detection frames are arranged from near to far according to the time when the detection frames are obtained, and the detection frame at the first position is the detection frame of the current frame.

[0046] Step S103, generating predicted detection frame information of the current frame according to the detection frame information of the previous frame, and generating a second detection frame sequence of the current frame according to the predicted detection frame information of the current frame.

[0047] In one embodiment, predicted detection frame information of a current frame is generated based on detection frame information of a previous frame, and a second detection frame sequence of the current frame is generated based on the predicted detection frame information of the current frame, including: at an initial moment, the second detection frame sequence is the same as the first detection frame sequence; after the initial moment, the detection frame information of the previous frame is obtained, the predicted detection frame information of the next frame is generated based on the detection frame information of the previous frame, and the second detection frame sequence is generated based on the first detection frame sequence of the previous frame and the predicted detection frame information of the next frame.

[0048] For the first detection box sequence, a second detection box sequence is constructed. For a certain obstacle, the second detection box sequence is Trackbox1{box1_1, box1_2, box1_3}. For all obstacles in the current environment image, the second detection box sequence set is obtained as: TrackerSet={Trackbox11{box1_1, box1_2, box1_3}, Trackbox12{box2_1, box2_2}…}, where for the road graphics of consecutive frames, the frame confidence includes: TrackerConfidence={c1, c2…}. The confidence of the current frame is the predicted confidence obtained from the previous frame image.

[0049] At the initial moment, TrackerSet is empty. {box1_1, box1_2, box1_3} is a continuous sequence of historical frames of a certain detection frame, that is, the first detection frame sequence. Among them, box1_1 is the most recent moment, and TrackerConfidence expresses the confidence of a series of continuous historical frames, which is consistent with the length of TrackerSet, and one confidence corresponds to one obstacle. For the second detection frame sequence of the current frame, the first detection frame sequence of the previous moment is obtained, and the predicted detection frame of the current frame is predicted based on the first detection frame sequence of the previous moment to generate the current second detection frame sequence.

[0050] Step 104 , matching the first detection frame sequence and the second detection frame sequence of the same frame, and determining the target detection frame of the obstacle according to the matching result.

[0051] During the matching process, a predicted detection frame of the current frame is obtained according to the second detection frame sequence, and a detection frame of the current frame is obtained according to the first detection sequence; and detection frames of the same frame are compared.

[0052] In this embodiment, a first detection frame sequence and a second detection frame sequence of the same frame are matched, and a target detection frame of the obstacle is determined according to the matching result, including: acquiring detection frame information and predicted detection frame information of the same frame according to the first detection frame sequence and the second detection frame sequence; matching the detection frame information and the predicted detection frame information of the same frame to obtain a matching result; wherein the matching result includes: a first matching result, a second matching result, and a third matching result, wherein the first matching result is that the second detection frame sequence does not receive the detection frame information corresponding to the current frame; the second matching result is that the detection frame information and the predicted detection frame information of the same frame do not match; the third matching result is that the detection frame information and the predicted detection frame information of the same frame match; the second detection frame sequence is updated according to the matching result to obtain a third detection frame sequence of the obstacle; and the target detection frame of the obstacle is determined according to the third detection frame sequence corresponding to all obstacles in the environment image.

[0053] Specifically, in this embodiment, the distance between the first detection sequence DetBox and the second detection sequence TrackerBox is calculated, that is, according to the predicted detection box of the current frame and the detection box of the current frame, the distance from DetSet and TrackerBox is calculated and Hungarian matching is performed to determine whether the predicted detection box and the detection box match, thereby obtaining three matching results.

[0054] In this embodiment, the first matching result, the second detection frame sequence TrackerSet does not match the first detection frame sequence DetSet. The first matching result means that there is only a predicted trajectory of the obstacle without a corresponding actual obstacle, which means that the obstacle frame corresponding to the current frame has not been received. At this time, the obstacle may have disappeared, so the obstacle information of the first matching result will be gradually reduced. The second matching result, the first detection frame sequence DetBox does not match the second detection frame sequence TrackerBox, which means that there is a detection frame of the current frame that does not meet the historical prediction. At this time, a new obstacle is likely to appear. Since there is no trajectory before the new obstacle, the new obstacle cannot be matched. For the third matching result, the second detection frame sequence TrackerBox successfully establishes a match with the first detection frame sequence DetBox, which means that the detection result of the current frame is reliable, and the continuous environment image continues to track the detection frame of the current frame.

[0055] In one embodiment, updating the second detection frame sequence according to the matching result includes: when the matching result is the first matching result; obtaining the confidence of the second detection frame sequence of the current frame; obtaining a preset confidence difference value, and reducing the confidence of the second detection frame sequence according to the preset confidence difference value; obtaining the confidence of the updated second detection frame sequence, and when the confidence of the updated second detection frame sequence is less than a preset threshold value, deleting the second detection frame sequence corresponding to the obstacle.

[0056] For the first matching result, TrackerBox does not match DetBox, for example, {box2_1, box2_2} does not match the detection box. At this time, the confidence c2 corresponding to the second detection box sequence is updated as shown in formula (3): Formula (3); Among them, when the confidence of c2 is less than 0, the trajectory is deleted.

[0057] In one embodiment, the second detection frame sequence is updated according to the matching result, including when the matching result is the second matching result; obtaining unmatched detection frame information, dividing the unmatched detection frame information into a first detection frame sequence of the newly added obstacle; and generating a second detection frame sequence of the newly added obstacle according to the first detection frame sequence of the newly added obstacle.

[0058] For the second matching result, DetBox does not match TrackerBox, such as DetBox1. At this time, a new track {DetBox1} is created, and the confidence of the track is Dc1, which is the confidence of the current detection box information. The output is DetBox1. No correction is made, and only one track is created.

[0059] In one embodiment, updating the second detection frame sequence according to the matching result includes: when the matching result is the third matching result; obtaining a first confidence of the first detection frame sequence and a second confidence of the second detection frame sequence; calculating an updated confidence according to the first confidence and the second confidence; obtaining the detection frame information of the current frame, replacing the predicted detection frame information of the current frame in the second detection frame sequence with the detection frame information of the current frame, and updating the second detection frame sequence.

[0060] For the third matching result, for example, for the first obstacle, the second detection box sequence TrackerBox1 matches the first detection box sequence DetBox1 successfully, and the corresponding confidence of TrackerBox1 is TrackerConfidence c1; the update formula of c1 is as shown in formula (4): c1 = c1 + Iou Dc2 formula (4); Where: Iou refers to the intersection-and-union ratio of the predicted detection box box1_1 of the current frame of TrackerBox1 and the detection box DetBox2 of the current frame. D is the detection box, c2 is the detection box confidence of the first detection box sequence, and DetBox2 is inserted into the head of the current continuous frame {box1_1, box1_2, box1_3}, that is, {DetBox2, box1_1, box1_2, box1_3}, that is, according to the inspection box information of the current frame and the detection box prediction information of the current frame, the intersection-and-union ratio is calculated, and according to the intersection-and-union ratio, the confidence of the second detection box sequence is updated, and the detection box information of the current frame replaces the predicted detection box information of the current frame in the second detection box sequence. At the same time, for the current second detection box sequence, the actual detection box information replaces the predicted detection box information.

[0061] Step 105 , matching the first detection frame sequence and the second detection frame sequence of the same frame, and determining the target detection frame of the obstacle according to the matching result.

[0062] The target detection frame of the obstacle is determined according to the updated second detection frame sequence. When the environment image includes multiple second detection frame sequences, each detection frame sequence corresponds to an obstacle.

[0063] After the target detection frame is obtained in step 105, the target detection frame is transformed from the world coordinate system to the vehicle coordinate system and outputted. The formulas are formula (5) and formula (6), including: Formula (5); Formula (6); Among them, x, y, z, sita are variables of the vehicle coordinate system, and x1, y1, z1, sita1 are variables of the world coordinate system.

[0064] This application obtains the first detection frame sequence corresponding to the obstacle through the environmental images of continuous frames to realize the detection frame trajectory tracking of the same obstacle in continuous frames. At the same time, this application constructs a second detection frame sequence of the obstacle by predicting the trajectory information of the current frame, and the second detection frame sequence is used to further track the trajectory information of the obstacle. By matching the first detection frame sequence and the second detection frame sequence of the same frame, the detection frame trajectory can be verified while tracking the detection frame trajectory. According to the matching results, it can be determined whether the obstacle has moved or disappeared, thereby avoiding the acquisition of the detection frame information of a fixed object and improving the accuracy of the detection frame information.

[0065] Second, as Figure 3 As shown, a detection frame acquisition device 30 is provided, including: an acquisition unit 301, a recognition unit 302, a detection unit 303, a prediction unit 304 and a result unit 305. Figure 3 Specifically include: The acquisition unit 301 is used to acquire continuous frames of environmental images during the driving of the vehicle.

[0066] The recognition unit 302 is used to perform obstacle recognition on each frame of the environment image to obtain detection frame information of the obstacle in each frame of the environment image.

[0067] The detection unit 303 is used to generate a first detection frame sequence of the current frame according to the acquisition time of the detection frame information and the detection frame information.

[0068] The prediction unit 304 is used to generate predicted detection frame information of the current frame according to the detection frame information of the previous frame, and generate a second detection frame sequence of the current frame according to the predicted detection frame information of the current frame.

[0069] The result unit 305 is used to match the first detection frame sequence and the second detection frame sequence of the same frame, and determine the target detection frame of the obstacle according to the matching result.

[0070] In one embodiment, obstacle recognition is performed on each frame of the environmental image to obtain detection frame information of the obstacle in each frame of the environmental image, including: based on a detection frame detection algorithm, obstacle recognition is performed on each frame of the environmental image to obtain a first type of detection frame corresponding to the obstacle and the confidence of the first type of detection frame; based on the confidence of the first type of detection frame, the credibility of the first type of detection frame is calculated; the credibility threshold of the first type of detection frame is obtained, and the first type of detection frame that does not meet the credibility threshold is eliminated to obtain a second type of detection frame; the positioning signal corresponding to the environmental image is obtained, and based on the second type of detection frame and the positioning signal, the coordinates of the second type of detection frame are converted into the world coordinate system to obtain a third type of detection frame; and the detection frame information is generated according to the third type of detection frame and the confidence of the third type of detection frame.

[0071] In one embodiment, predicted detection frame information of a current frame is generated based on detection frame information of a previous frame, and a second detection frame sequence of the current frame is generated based on the predicted detection frame information of the current frame, including: at an initial moment, the second detection frame sequence is the same as the first detection frame sequence; after the initial moment, the detection frame information of the previous frame is obtained, the predicted detection frame information of the next frame is generated based on the detection frame information of the previous frame, and the second detection frame sequence is generated based on the first detection frame sequence of the previous frame and the predicted detection frame information of the next frame.

[0072] In one embodiment, a first detection frame sequence and a second detection frame sequence of the same frame are matched, and a target detection frame of the obstacle is determined according to the matching result, including: obtaining detection frame information and predicted detection frame information of the same frame according to the first detection frame sequence and the second detection frame sequence; matching the detection frame information and the predicted detection frame information of the same frame to obtain a matching result; wherein the matching result includes: a first matching result, a second matching result, and a third matching result, wherein the first matching result is that the second detection frame sequence does not receive the detection frame information corresponding to the current frame; the second matching result is that the detection frame information and the predicted detection frame information of the same frame do not match; the third matching result is that the detection frame information and the predicted detection frame information of the same frame match; the second detection frame sequence is updated according to the matching result to obtain a third detection frame sequence of the obstacle; and the target detection frame of the obstacle is determined according to the third detection frame sequence corresponding to all obstacles in the environment image.

[0073] In one embodiment, updating the second detection frame sequence according to the matching result includes: when the matching result is the first matching result; obtaining the confidence of the second detection frame sequence of the current frame; obtaining a preset confidence difference value, and reducing the confidence of the second detection frame sequence according to the preset confidence difference value; obtaining the confidence of the updated second detection frame sequence, and when the confidence of the updated second detection frame sequence is less than a preset threshold value, deleting the second detection frame sequence corresponding to the obstacle.

[0074] In one embodiment, updating the second detection frame sequence according to the matching result includes: when the matching result is the second matching result; obtaining unmatched detection frame information, dividing the unmatched detection frame information into a first detection frame sequence of the newly added obstacle; and generating a second detection frame sequence of the newly added obstacle according to the first detection frame sequence of the newly added obstacle.

[0075] In one embodiment, updating the second detection frame sequence according to the matching result includes: when the matching result is the third matching result; obtaining a first confidence of the first detection frame sequence and a second confidence of the second detection frame sequence; calculating an updated confidence according to the first confidence and the second confidence; obtaining the detection frame information of the current frame, replacing the predicted detection frame information of the current frame in the second detection frame sequence with the detection frame information of the current frame, and updating the second detection frame sequence.

[0076] Please refer to Figure 4 , Figure 4 This is a schematic diagram of an embodiment of an electronic device of the present application.

[0077] The electronic device 100 includes a memory 20, a processor 30, and a computer program 40 stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program 40, the steps in the above-mentioned detection frame acquisition method embodiment are implemented, for example Figure 1 Steps 101 to 105 shown, or, Figure 2 Steps 201 to 205 are shown.

[0078] Exemplarily, the computer program 40 can also be divided into one or more modules / units, one or more modules / units are stored in the memory 20 and executed by the processor 30. One or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 40 in the electronic device 100. For example, it can be divided into the acquisition unit 301, the recognition unit 302, the detection unit 303, the prediction unit 304 and the result unit 305 shown.

[0079] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation of the electronic device 100. The electronic device 100 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 100 may also include input and output devices, network access devices, buses, etc.

[0080] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, a single-chip microcomputer, or the processor 30 may also be any conventional processor, etc.

[0081] The memory 20 can be used to store the computer program 40 and / or the module / unit. The processor 30 realizes various functions of the electronic device 100 by running or executing the computer program and / or the module / unit stored in the memory 20 and calling the data stored in the memory 20. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0082] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

Claims

1. A detection frame acquisition method, characterized in that: include: During the driving process of the vehicle, continuous frames of environmental images are acquired; Perform obstacle recognition on each frame of the environment image to obtain detection frame information of the obstacle in each frame of the environment image; Generate a first detection frame sequence of the current frame according to the acquisition time of the detection frame information and the detection frame information; Generate predicted detection frame information of the current frame according to the detection frame information of the previous frame, and generate a second detection frame sequence of the current frame according to the predicted detection frame information of the current frame; The first detection frame sequence and the second detection frame sequence of the same frame are matched, and a target detection frame of the obstacle is determined according to the matching result.

2. The detection frame acquisition method according to claim 1, characterized in that: The step of performing obstacle recognition on each frame of the environment image to obtain detection frame information of obstacles in each frame of the environment image includes: Based on the detection frame detection algorithm, obstacle recognition is performed on each frame of the environment image to obtain a first type of detection frame corresponding to the obstacle and a confidence level of the first type of detection frame; Calculating the credibility of the first type of detection box based on the confidence of the first type of detection box; Obtaining a credibility threshold of the first category detection frame, removing the first category detection frame that does not meet the credibility threshold, and obtaining the second category detection frame; Acquire a positioning signal corresponding to the environment image, and based on the second type of detection frame and the positioning signal, convert the coordinates of the second type of detection frame into a world coordinate system to obtain a third type of detection frame; The detection frame information is generated according to the third category detection frame and the confidence of the third category detection frame.

3. The detection frame acquisition method according to claim 1, characterized in that: The generating the predicted detection frame information of the current frame according to the detection frame information of the previous frame, and generating the second detection frame sequence of the current frame according to the predicted detection frame information of the current frame, comprises: At an initial moment, the second detection frame sequence is the same as the first detection frame sequence; After the initial moment, the detection frame information of the previous frame is obtained, the predicted detection frame information of the next frame is generated according to the detection frame information of the previous frame, and the second detection frame sequence is generated according to the first detection frame sequence of the previous frame and the predicted detection frame information of the next frame.

4. The detection frame acquisition method according to claim 1, characterized in that: The matching the first detection frame sequence and the second detection frame sequence of the same frame, and determining the target detection frame of the obstacle according to the matching result, includes: Acquire detection frame information and predicted detection frame information of the same frame according to the first detection frame sequence and the second detection frame sequence; Matching the detection frame information of the same frame with the predicted detection frame information to obtain the matching result; The matching result includes: a first matching result, a second matching result and a third matching result, the first matching result is that the second detection frame sequence does not receive the detection frame information corresponding to the current frame; the second matching result is that the detection frame information of the same frame and the predicted detection frame information do not match; the third matching result is that the detection frame information of the same frame and the predicted detection frame information match; Update the second detection frame sequence according to the matching result to obtain a third detection frame sequence of the obstacle; Determine a target detection frame of the obstacle according to a third detection frame sequence corresponding to all obstacles in the environment image.

5. The detection frame acquisition method according to claim 4, characterized in that: The updating of the second detection frame sequence according to the matching result includes: In a case where the matching result is the first matching result; Obtain the confidence of the second detection frame sequence of the current frame; Acquire a preset confidence difference value, and reduce the confidence of the second detection frame sequence according to the preset confidence difference value; The confidence of the updated second detection frame sequence is obtained, and when the confidence of the updated second detection frame sequence is less than a preset threshold, the second detection frame sequence corresponding to the obstacle is deleted.

6. The detection frame acquisition method according to claim 4, characterized in that: The updating of the second detection frame sequence according to the matching result includes: In a case where the matching result is the second matching result; Acquire unmatched detection frame information, and divide the unmatched detection frame information into a first detection frame sequence of newly added obstacles; A second detection frame sequence of the newly added obstacle is generated according to the first detection frame sequence of the newly added obstacle.

7. The detection frame acquisition method according to claim 4, characterized in that: The updating of the second detection frame sequence according to the matching result includes: In a case where the matching result is the third matching result; Obtaining a first confidence of the first detection frame sequence and a second confidence of the second detection frame sequence; Calculating the updated confidence level according to the first confidence level and the second confidence level; Obtain detection frame information of the current frame, replace the predicted detection frame information of the current frame in the second detection frame sequence with the detection frame information of the current frame, and update the second detection frame sequence.

8. A detection frame acquisition device, characterized in that: include: An acquisition unit, used for acquiring continuous frames of environmental images during the driving of the vehicle; An identification unit, used to identify obstacles in each frame of the environment image, and obtain detection frame information of obstacles in each frame of the environment image; A detection unit, configured to generate a first detection frame sequence of the current frame according to the acquisition time of the detection frame information and the detection frame information; A prediction unit, configured to generate predicted detection frame information of a current frame according to the detection frame information of a previous frame, and generate a second detection frame sequence of the current frame according to the predicted detection frame information of the current frame; A result unit is used to match the first detection frame sequence and the second detection frame sequence of the same frame, and determine the target detection frame of the obstacle according to the matching result.

9. An electronic device, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the detection frame acquisition method according to any one of claims 1 to claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a car, the car executes the detection frame acquisition method according to any one of claims 1 to 7.

Images