A monocular vision-based forward collision object recognition method and system

Abstract

The application relates to the technical field of collision object recognition, in particular to a forward collision object recognition method and system based on monocular vision, which comprises the following steps: acquiring a forward vision image and carrying out object recognition to form a labeled image and add the labeled image into a labeled image sequence; adopting an optical flow method to match a detected object in a current frame of the labeled image with a detected object in a historical frame of the labeled image to form a marker sequence; combining an external parameter matrix to respectively calculate the relative acceleration of the detected object in the current frame; and carrying out threshold value discrimination according to the relative acceleration and generating a recognition result. In view of the problem that the forward collision algorithm in the prior art has poor target tracking effect, the optical flow method is introduced to frame-by-frame match the detected object, so that a complete marker sequence corresponding to the detected object is established, and then the relative acceleration of the object relative to the camera is calculated according to each marker sequence in combination with the external parameter matrix to carry out discrimination, so that a more accurate discrimination process of the acceleration is realized.

Description

Technical Field

[0001] This invention relates to the field of collision object recognition technology, and specifically to a forward collision object recognition method and system based on monocular vision. Background Technology

[0002] Forward collision detection is a crucial function in driver assistance systems. It refers to the process by which a vehicle uses various sensors, such as LiDAR, image sensors, and millimeter-wave radar, to detect moving or stationary objects in front of it and predict whether a collision will occur. This process typically relies on specific sensors to perceive the surrounding environment, followed by specific algorithms that track the target and calculate its acceleration to complete the detection.

[0003] Similar technical solutions already exist in the existing technology.

[0004] For example, patent application CN202010423008.4 discloses a method for forward collision warning of vehicles. This method uses C-V2X technology to monitor for forward collision warnings of vehicles traveling in the same lane and / or adjacent lanes while the vehicle is driving on the road. The method includes: the vehicle detecting the collision risk of vehicles traveling in the same lane and / or adjacent lanes providing turn signals based on V2I information from roadside units and V2V information from vehicles traveling in the same lane and / or adjacent lanes providing turn signals; if a collision risk exists, the collision time is calculated and the minimum collision time is selected; the current road driving conditions and the corresponding forward collision probability versus collision time correlation curve are obtained, and the forward collision probability is searched in the correlation curve based on the minimum collision time; once the forward collision probability is determined to be greater than or equal to a corresponding preset threshold, a forward collision warning is issued to the driver of the vehicle. This method, based on the accurate identification of target vehicles under various road shapes using C-V2X technology, achieves forward collision warnings, greatly improving vehicle driving safety.

[0005] For example, patent application CN202510438603.8 discloses a forward collision avoidance warning method and system based on lane recognition using millimeter-wave radar, relating to the field of intelligent driving technology. The method includes: acquiring point cloud information transmitted by millimeter-wave radar and driving data corresponding to the current vehicle; generating target information based on the point cloud information, whereby the target information indicates at least one target within the detection range of the millimeter-wave radar; combining the target information and motion data to generate a predicted collision result between the current vehicle and the target; and generating collision avoidance warning information based on the predicted collision result. By filtering and judging the target's lane, speed, and pose information based on the point cloud clustering and tracking results, the distance to the vehicle ahead and the collision time are accurately estimated, and appropriate warning measures are taken, further improving the reliability and anti-interference capability of the results.

[0006] However, in actual implementation, the inventors found that this type of technical solution usually tracks a single target, and when identifying multiple targets, there are tracking errors that make it impossible to accurately calculate acceleration. Summary of the Invention

[0007] To address the aforementioned problems in the existing technology, a forward collision object recognition method based on monocular vision is provided. On the other hand, a system for implementing this method is also provided.

[0008] The specific technical solution is as follows: A forward collision object recognition method based on monocular vision includes: Step S1: Acquire the forward-facing visual image and perform object recognition to form an annotated image, which is then added to the annotated image sequence; Step S2: Using optical flow, the detected objects in the current frame's labeled image are matched with the detected objects in the labeled images of historical frames to form a marker sequence; Step S3: For each marker sequence, calculate the relative acceleration of the detected object in the current frame using the extrinsic parameter matrix; Step S4: Perform threshold discrimination based on the relative acceleration and generate recognition results.

[0009] On the other hand, step S1 includes: Step S11: Acquire the forward visual image and use a pre-trained convolutional network model to identify the forward visual image to obtain the detected object; Step S12: Mark the detected object on the forward-looking visual image to form the marked image; Step S13: Add the labeled image to the labeled image sequence.

[0010] On the other hand, step S2 includes: Step S21: Perform optical flow analysis on multiple frames of the detected images from the labeled sequence to obtain the optical flow field vector; Step S22: Obtain the detected object in the historical frame, and predict the identification image of the current frame according to the optical flow field vector to obtain the search direction of the associated object; Step S23: Based on the search direction of the associated objects, find the matching detected objects and establish a marker correspondence relationship; Step S24: Add the marker correspondence to the marker sequence.

[0011] On the other hand, step S3 includes: Step S31: Extract the three most recent frames from the marker sequence as detection frames, and extract the bottom edge of the detected object image for each detection frame; Step S32: Map the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates; Step S33: Estimate the relative acceleration based on all the ground coordinate system coordinates.

[0012] On the other hand, step S4 includes: Step S41: Obtain the relative acceleration of multiple frames and predict the predicted relative acceleration of the next frame; Step S42: Compare the predicted relative acceleration with the acceleration threshold to obtain the identification result.

[0013] A forward collision object recognition system based on monocular vision is used to implement the aforementioned forward collision object recognition method; The forward collision object recognition system includes: The acquisition module acquires forward-looking visual images and performs object recognition to form labeled images, which are then added to the labeled image sequence. A matching module is connected to the acquisition module; The matching module uses optical flow to match the detected objects in the current frame's labeled image with the detected objects in the labeled images of historical frames to form a marker sequence. An acceleration calculation module is connected to the matching module; The acceleration calculation module calculates the relative acceleration of the detected object in the current frame for each marker sequence in combination with the extrinsic parameter matrix; The discrimination module is connected to the acceleration calculation module; The discrimination module performs threshold discrimination based on the relative acceleration and generates recognition results.

[0014] On the other hand, the acquisition module includes: The recognition module acquires the forward visual image and uses a pre-trained convolutional network model to recognize the forward visual image to obtain the detected object. Annotation module, which is connected to the recognition module; The annotation module annotates the detected object on the forward-looking visual image to form the annotation image; Add a module, which is connected to the annotation module; The adding module adds the labeled image to the labeled image sequence.

[0015] On the other hand, the matching module includes: An optical flow module is provided, which performs optical flow analysis on multiple frames of the detected images from the labeled sequence to obtain an optical flow field vector. A prediction module, which is connected to the optical flow module; The prediction module acquires the detected objects in historical frames and predicts the identification image of the current frame based on the optical flow field vector to obtain the search direction of the associated objects. A search module, which is connected to the prediction module; The search module finds the matching detected objects according to the search direction of the associated objects and establishes a marker correspondence relationship; A recording module, which is connected to the search module; The recording module adds the marker correspondence to the marker sequence.

[0016] On the other hand, the acceleration calculation module includes: An extraction module extracts the three most recent frames from the marker sequence as detection frames, and extracts the bottom edge of the detected object image for each detection frame. A mapping module, which is connected to the extraction module; The mapping module maps the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates. An estimation module, which is connected to the mapping module; The estimation module estimates the relative acceleration based on all the coordinates of the ground coordinate system.

[0017] On the other hand, the discrimination module includes: An acceleration prediction module acquires the relative acceleration of multiple frames and predicts the predicted relative acceleration of the next frame. A comparison module, which is connected to the acceleration prediction module; The comparison module compares the predicted relative acceleration with an acceleration threshold to obtain the identification result.

[0018] The above technical solution has the following advantages or beneficial effects: To address the issue of poor target tracking performance in existing forward collision algorithms, an optical flow method is introduced to perform frame-by-frame matching of the detected object, thereby establishing a complete sequence of markers corresponding to the detected object. Then, the relative acceleration of the object relative to the camera is calculated based on each marker sequence and the extrinsic parameter matrix for discrimination, thus achieving a more accurate discrimination process for acceleration. Attached Figure Description

[0019] Embodiments of the invention will be described more fully with reference to the accompanying drawings. However, the drawings are for illustration and explanation only and do not constitute a limitation on the scope of the invention.

[0020] Figure 1 This is an overall schematic diagram of an embodiment of the present invention; Figure 2 This is a schematic diagram of step S1 in an embodiment of the present invention; Figure 3 This is a schematic diagram of step S2 in an embodiment of the present invention; Figure 4 This is a schematic diagram of step S3 in an embodiment of the present invention; Figure 5 This is a schematic diagram of step S4 in an embodiment of the present invention; Figure 6 This is a schematic diagram of the system in an embodiment of the present invention; Figure 7 This is a schematic diagram of the acquisition module in an embodiment of the present invention; Figure 8 This is a schematic diagram of the matching module in an embodiment of the present invention; Figure 9 This is a schematic diagram of the acceleration calculation module in an embodiment of the present invention; Figure 10 This is a schematic diagram of the discrimination module in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0024] This invention includes: A forward collision object recognition method based on monocular vision, such as Figure 1 As shown, it includes: Step S1: Acquire the forward-facing visual image and perform object recognition to form an annotated image, which is then added to the annotated image sequence; Step S2: Using optical flow, the detected objects in the current frame's labeled image are matched with the detected objects in the historical frame's labeled image to form a marker sequence. Step S3: For each marker sequence, calculate the relative acceleration of the detected object in the current frame using the extrinsic parameter matrix; Step S4: Determine the threshold based on relative acceleration and generate the recognition result.

[0025] Specifically, to address the issue of poor target tracking performance of existing forward collision algorithms, an optical flow method is introduced to perform frame-by-frame matching of the detected object, thereby establishing a complete sequence of markers corresponding to the detected object. Then, the relative acceleration of the object relative to the camera is calculated based on each marker sequence in conjunction with the extrinsic parameter matrix, thus achieving a more accurate discrimination process for acceleration.

[0026] In one embodiment, such as Figure 2 As shown, step S1 includes: Step S11: Acquire the forward visual image and use a pre-trained convolutional network model to identify the detected object from the forward visual image; Step S12: Mark the detected objects on the forward-looking visual image to form a labeled image; Step S13: Add the labeled images to the labeled image sequence.

[0027] Specifically, in order to effectively identify moving objects, in this embodiment, for the forward visual image acquired by the camera, a pre-trained convolutional network model is first used to segment the target objects in the image, including vehicles, two-wheeled vehicles, pedestrians, roadblocks and other obstacles, so as to obtain the detected objects.

[0028] The convolutional network model is a U-net structure. The encoder extracts features from the image through multiple convolutional layers with different receptive fields to obtain a high-dimensional feature map. Then, in the decoder, a series of inverse upsampling layers are used to restore the features of the high-dimensional feature map, thereby restoring the segmented detected object.

[0029] To improve the accuracy of segmentation, the above convolutional network model also establishes skip connections between convolutional layers and upsampling layers of the same receptive field scale through residual connections to input the original information into the upsampling layer, thus avoiding model overfitting.

[0030] Subsequently, the bounding boxes corresponding to the detected objects output by the model are added to the forward visual image to form labeled images, and then the labeled images are added to the labeled image sequence for subsequent identification.

[0031] In one embodiment, such as Figure 3 As shown, step S2 includes: Step S21: Perform optical flow analysis on multiple frames of detected images from the self-labeled sequence to obtain the optical flow field vector; Step S22: Obtain the detected objects in the historical frames, and predict the identification image of the current frame based on the optical flow field vector to obtain the search direction of the associated objects; Step S23: Based on the search direction of associated objects, find the matching detected objects and establish the correspondence between the markers; Step S24: Add the marker correspondence to the marker sequence.

[0032] Specifically, to achieve effective tracking of moving objects across multiple frames, this embodiment, after acquiring the labeled sequence, first performs a pixel-by-pixel optical flow analysis on the detected image to obtain the optical flow field vector of each pixel in the image. For vehicles in motion, the optical flow field in their images exhibits significant regional differences: near-uniformly moving background areas, such as lanes and the sky, and areas with speeds differing from the background, such as other moving vehicles or two-wheeled vehicles. This results in a clear regionalization phenomenon in the pixel-by-pixel optical flow field vector.

[0033] Subsequently, pre-labeled bounding boxes are identified from historical frames to obtain the detected objects, and these bounding boxes are overlaid with optical flow field vectors to obtain the optical flow field vectors in the corresponding partitions. Based on the optical flow field vectors in the partitions, the labeled image of the current frame is predicted to obtain the search direction of the associated objects.

[0034] The associated object search direction is the mean value of the optical flow field vector in the partition. It has a starting position and points to a specific orientation and length in the image. Then, based on the associated object search direction, the current frame's label image is searched to obtain the corresponding bounding box, thereby completing the matching process, obtaining the matched detected object, and establishing the marker correspondence. Finally, the bounding boxes of the markers in multiple frames are transferred to the original forward monocular vision image to form a marker image corresponding to a single detected object. Then, the marker image is added to the marker sequence.

[0035] In one embodiment, such as Figure 4 As shown, step S3 includes: Step S31: Extract the three most recent frames from the marker sequence as detection frames, and extract the bottom edge of the detected object image for each detection frame; Step S32: Map the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates; Step S33: Estimate the relative acceleration based on all ground coordinate system coordinates.

[0036] Specifically, considering that the pixel area occupied by the detected object varies greatly at different depths, and that the projected area of ​​different types of detected objects, such as heavy vehicles, small vehicles and two-wheeled vehicles, also varies greatly, the traditional visual estimation algorithm based on the object width is not applicable.

[0037] To address this issue, the estimation algorithm in this embodiment is modified to identify only the lowest point of the detected object, typically the projection position of the rear wheel of a two-wheeled vehicle or the rear bumper of a car relative to its own depth. Then, relying on a known extrinsic parameter matrix, i.e., an extrinsic parameter matrix calibrated according to the camera's installation height, pitch angle, and focal length, this part of the position is converted into a real ground coordinate system position and the acceleration is estimated.

[0038] Specifically, to achieve the estimation process, the images of the three most recent consecutive frames are needed to provide a continuous acceleration estimation process. For the three detection frames extracted from the marker sequence, the bottom edge of the bounding box in the marker image is obtained respectively. Then, the extrinsic parameter matrix is ​​used to map the bottom edge of the image from the pixel coordinate system to the ground coordinate system to obtain the ground coordinate system coordinates on the three consecutive frames.

[0039] Finally, based on the changes in all ground coordinates across the three frames, and combined with the frame sampling time, the relative acceleration relative to the vehicle itself is estimated.

[0040] In one embodiment, such as Figure 5 As shown, step S4 includes: Step S41: Obtain the relative acceleration of multiple frames and predict the predicted relative acceleration of the next frame; Step S42: Compare the predicted relative acceleration with the acceleration threshold to obtain the identification result.

[0041] Specifically, in order to achieve a better smoothing filtering effect, in this embodiment, after calculating the relative acceleration, the relative acceleration of multiple frames is obtained and input into a Kalman filter for smoothing processing to predict the relative acceleration of the next frame. Then, the predicted relative acceleration is compared with the acceleration threshold to obtain the identification result, so as to determine whether there is a collision risk and provide it to the corresponding AEB system.

[0042] A forward collision object recognition system based on monocular vision is used to implement the aforementioned forward collision object recognition method; like Figure 6 As shown, the forward collision detection system includes: Acquisition module 1 acquires forward visual images and performs object recognition to form labeled images, which are then added to the labeled image sequence. Matching module 2 is connected to acquisition module 1; In the self-annotated image sequence, the matching module 2 uses optical flow to match the detected objects in the current frame's annotated image with the detected objects in the annotated images of historical frames to form a marker sequence; Acceleration calculation module 3 is connected to matching module 2; Acceleration calculation module 3 calculates the relative acceleration of the detected object in the current frame for each marker sequence in combination with the extrinsic parameter matrix; Discrimination module 4 is connected to acceleration calculation module 3; The discrimination module 4 performs threshold discrimination based on relative acceleration and generates recognition results.

[0043] Specifically, to address the issue of poor target tracking performance of existing forward collision algorithms, an optical flow method is introduced to perform frame-by-frame matching of the detected object, thereby establishing a complete sequence of markers corresponding to the detected object. Then, the relative acceleration of the object relative to the camera is calculated based on each marker sequence in conjunction with the extrinsic parameter matrix, thus achieving a more accurate discrimination process for acceleration.

[0044] In one embodiment, such as Figure 7 As shown, the acquisition module 1 includes: The recognition module 11 acquires the forward visual image and uses a pre-trained convolutional network model to recognize the forward visual image in order to obtain the detected object. Annotation module 12, which is connected to recognition module 11; The annotation module 12 annotates the detected objects on the forward-looking visual image to form an annotated image; Add module 13, and connect module 13 to annotation module 12; Add module 13 to add the labeled images to the labeled image sequence.

[0045] Specifically, in order to effectively identify moving objects, in this embodiment, for the forward visual image acquired by the camera, a pre-trained convolutional network model is first used to segment the target objects in the image, including vehicles, two-wheeled vehicles, pedestrians, roadblocks and other obstacles, so as to obtain the detected objects.

[0046] The convolutional network model is a U-net structure. The encoder extracts features from the image through multiple convolutional layers with different receptive fields to obtain a high-dimensional feature map. Then, in the decoder, a series of inverse upsampling layers are used to restore the features of the high-dimensional feature map, thereby restoring the segmented detected object.

[0047] To improve the accuracy of segmentation, the above convolutional network model also establishes skip connections between convolutional layers and upsampling layers of the same receptive field scale through residual connections to input the original information into the upsampling layer, thus avoiding model overfitting.

[0048] Subsequently, the bounding boxes corresponding to the detected objects output by the model are added to the forward visual image to form labeled images, and then the labeled images are added to the labeled image sequence for subsequent identification.

[0049] In one embodiment, such as Figure 8 As shown, matching module 2 includes: Optical flow module 21 performs optical flow analysis on multiple frames of detection images in the self-labeled sequence to obtain optical flow field vectors; Prediction module 22, prediction module 22 is connected to optical flow module 21; The prediction module 22 acquires the detected objects in the historical frames and predicts the identification image of the current frame based on the optical flow field vector to obtain the search direction of the associated objects; Search module 23, which is connected to prediction module 22; The search module 23 searches for matching detected objects based on the search direction of associated objects and establishes a correspondence between markers; Recording module 24, which is connected to search module 23; Recording module 24 adds the marker correspondence to the marker sequence.

[0050] Specifically, to achieve effective tracking of moving objects across multiple frames, this embodiment, after acquiring the labeled sequence, first performs a pixel-by-pixel optical flow analysis on the detected image to obtain the optical flow field vector of each pixel in the image. For vehicles in motion, the optical flow field in their images exhibits significant regional differences: near-uniformly moving background areas, such as lanes and the sky, and areas with speeds differing from the background, such as other moving vehicles or two-wheeled vehicles. This results in a clear regionalization phenomenon in the pixel-by-pixel optical flow field vector.

[0051] Subsequently, pre-labeled bounding boxes are identified from historical frames to obtain the detected objects, and these bounding boxes are overlaid with optical flow field vectors to obtain the optical flow field vectors in the corresponding partitions. Based on the optical flow field vectors in the partitions, the labeled image of the current frame is predicted to obtain the search direction of the associated objects.

[0052] The associated object search direction is the mean value of the optical flow field vector in the partition. It has a starting position and points to a specific orientation and length in the image. Then, based on the associated object search direction, the current frame's label image is searched to obtain the corresponding bounding box, thereby completing the matching process, obtaining the matched detected object, and establishing the marker correspondence. Finally, the bounding boxes of the markers in multiple frames are transferred to the original forward monocular vision image to form a marker image corresponding to a single detected object. Then, the marker image is added to the marker sequence.

[0053] In one embodiment, such as Figure 9 As shown, the acceleration calculation module 3 includes: Extraction module 31 extracts the three most recent frames from the marker sequence as detection frames, and extracts the bottom edge of the detected object image for each detection frame. Mapping module 32, which is connected to extraction module 31; Mapping module 32 maps the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates; Estimation module 33, which is connected to mapping module 32; The estimation module 33 estimates the relative acceleration based on all ground coordinate system coordinates.

[0054] Specifically, considering that the pixel area occupied by the detected object varies greatly at different depths, and that the projected area of ​​different types of detected objects, such as heavy vehicles, small vehicles and two-wheeled vehicles, also varies greatly, the traditional visual estimation algorithm based on the object width is not applicable.

[0055] To address this issue, the estimation algorithm in this embodiment is modified to identify only the lowest point of the detected object, typically the projection position of the rear wheel of a two-wheeled vehicle or the rear bumper of a car relative to its own depth. Then, relying on a known extrinsic parameter matrix, i.e., an extrinsic parameter matrix calibrated according to the camera's installation height, pitch angle, and focal length, this part of the position is converted into a real ground coordinate system position and the acceleration is estimated.

[0056] Specifically, to achieve the estimation process, the images of the three most recent consecutive frames are needed to provide a continuous acceleration estimation process. For the three detection frames extracted from the marker sequence, the bottom edge of the bounding box in the marker image is obtained respectively. Then, the extrinsic parameter matrix is ​​used to map the bottom edge of the image from the pixel coordinate system to the ground coordinate system to obtain the ground coordinate system coordinates on the three consecutive frames.

[0057] Finally, based on the changes in all ground coordinates across the three frames, and combined with the frame sampling time, the relative acceleration relative to the vehicle itself is estimated.

[0058] In one embodiment, such as Figure 10 As shown, the discrimination module 4 includes: Acceleration prediction module 41 acquires the relative acceleration of multiple frames and predicts the predicted relative acceleration of the next frame; Comparison module 42, which is connected to acceleration prediction module 41; The comparison module 42 compares the predicted relative acceleration with the acceleration threshold to obtain the identification result.

[0059] Specifically, in order to achieve a better smoothing filtering effect, in this embodiment, after calculating the relative acceleration, the relative acceleration of multiple frames is obtained and input into a Kalman filter for smoothing processing to predict the relative acceleration of the next frame. Then, the predicted relative acceleration is compared with the acceleration threshold to obtain the identification result, so as to determine whether there is a collision risk and provide it to the corresponding AEB system.

[0060] Those skilled in the art will understand that various aspects, or possible implementations of various aspects, of the present invention can be embodied as systems, methods, or computer program products. Therefore, various aspects, or possible implementations of various aspects, of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuit," "module," or "system." Furthermore, various aspects, or possible implementations of various aspects, of the present invention can take the form of computer program products, which are computer instructions stored in memory.

[0061] The memory can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).

[0062] A processor in a computer reads computer instructions stored in memory, enabling the processor to execute the functional actions specified in each step or combination of steps in a flowchart; and to generate means for implementing the functional actions specified in each block or combination of blocks in a flowchart.

[0063] It should be understood that a processor in a computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components used to execute the aforementioned computer instructions.

[0064] Computer instructions may be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in some alternative implementations, the functions indicated by the steps in the flowchart or the blocks in the block diagram may not occur in the order shown in the diagram. For example, depending on the functions involved, two consecutive steps or blocks may actually be executed approximately simultaneously, or these blocks may sometimes be executed in reverse order.

[0065] Of course, in practical applications, the various components of a computer system are coupled together through a bus system. The bus system is used to enable communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0066] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying forward-facing collision objects based on monocular vision, characterized in that, include: Step S1: Acquire the forward-facing visual image and perform object recognition to form an annotated image, which is then added to the annotated image sequence; Step S2: Using optical flow, the detected objects in the current frame's labeled image are matched with the detected objects in the labeled images of historical frames to form a marker sequence; Step S3: For each marker sequence, calculate the relative acceleration of the detected object in the current frame using the extrinsic parameter matrix; Step S4: Perform threshold discrimination based on the relative acceleration and generate recognition results.

2. The forward collision object recognition method according to claim 1, characterized in that, Step S1 includes: Step S11: Acquire the forward visual image and use a pre-trained convolutional network model to identify the forward visual image to obtain the detected object; Step S12: Mark the detected object on the forward-looking visual image to form the marked image; Step S13: Add the labeled image to the labeled image sequence.

3. The forward collision object recognition method according to claim 1, characterized in that, Step S2 includes: Step S21: Perform optical flow analysis on multiple frames of the detected images from the labeled sequence to obtain the optical flow field vector; Step S22: Obtain the detected object in the historical frame, and predict the identification image of the current frame according to the optical flow field vector to obtain the search direction of the associated object; Step S23: Based on the search direction of the associated objects, find the matching detected objects and establish a marker correspondence relationship; Step S24: Add the marker correspondence to the marker sequence.

4. The forward collision object recognition method according to claim 1, characterized in that, Step S3 includes: Step S31: Extract the three most recent frames from the marker sequence as detection frames, and extract the bottom edge of the detected object image for each detection frame; Step S32: Map the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates; Step S33: Estimate the relative acceleration based on all the ground coordinate system coordinates.

5. The forward collision object recognition method according to claim 1, characterized in that, Step S4 includes: Step S41: Obtain the relative acceleration of multiple frames and predict the predicted relative acceleration of the next frame; Step S42: Compare the predicted relative acceleration with the acceleration threshold to obtain the identification result.

6. A forward collision object recognition system based on monocular vision, characterized in that, Used to implement the forward collision object identification method as described in any one of claims 1-5; The forward collision object recognition system includes: The acquisition module acquires forward-looking visual images and performs object recognition to form labeled images, which are then added to the labeled image sequence. A matching module is connected to the acquisition module; The matching module uses optical flow to match the detected objects in the current frame's labeled image with the detected objects in the labeled images of historical frames to form a marker sequence. An acceleration calculation module is connected to the matching module; The acceleration calculation module calculates the relative acceleration of the detected object in the current frame for each marker sequence in combination with the extrinsic parameter matrix; The discrimination module is connected to the acceleration calculation module; The discrimination module performs threshold discrimination based on the relative acceleration and generates recognition results.

7. The forward collision object recognition system according to claim 6, characterized in that, The acquisition module includes: The recognition module acquires the forward visual image and uses a pre-trained convolutional network model to recognize the forward visual image to obtain the detected object. Annotation module, which is connected to the recognition module; The annotation module annotates the detected object on the forward-looking visual image to form the annotation image; Add a module, which is connected to the annotation module; The adding module adds the labeled image to the labeled image sequence.

8. The forward collision object recognition system according to claim 6, characterized in that, The matching module includes: An optical flow module is provided, which performs optical flow analysis on multiple frames of the detected images from the labeled sequence to obtain an optical flow field vector. A prediction module, which is connected to the optical flow module; The prediction module acquires the detected objects in historical frames and predicts the identification image of the current frame based on the optical flow field vector to obtain the search direction of the associated objects. A search module, which is connected to the prediction module; The search module finds the matching detected objects according to the search direction of the associated objects and establishes a marker correspondence relationship; A recording module, which is connected to the search module; The recording module adds the marker correspondence to the marker sequence.

9. The forward collision object recognition system according to claim 6, characterized in that, The acceleration calculation module includes: An extraction module extracts the three most recent frames from the marker sequence as detection frames, and extracts the bottom edge of the detected object image for each detection frame. A mapping module, which is connected to the extraction module; The mapping module maps the bottom edge of the image to the ground coordinate system according to the extrinsic parameter matrix to obtain the ground coordinate system coordinates. An estimation module, which is connected to the mapping module; The estimation module estimates the relative acceleration based on all the coordinates of the ground coordinate system.

10. The forward collision object recognition system according to claim 6, characterized in that, The discrimination module includes: An acceleration prediction module acquires the relative acceleration of multiple frames and predicts the predicted relative acceleration of the next frame. A comparison module, which is connected to the acceleration prediction module; The comparison module compares the predicted relative acceleration with an acceleration threshold to obtain the identification result.

Images