Blind zone detection method, vehicle, readable storage medium and computer program product

Abstract

The invention discloses a blind area detection method, a vehicle, a readable storage medium and a computer program product. The blind area detection method comprises the following steps: acquiring a pedestrian image and detecting pedestrians in the pedestrian image; when a pedestrian enters a cart or a blind area behind a wall, collecting point cloud data; pedestrian joint features are extracted from the pedestrian image and are used as feature points; performing hierarchical fusion judgment on the point cloud data and the joint feature point data of the pedestrian; and if the target fusion is successful, matching the point cloud data with the pedestrian joint feature points, and fusing the confidence of the radar detection points to realize tracking of the pedestrian motion state. The method comprises the following steps: detecting a target, extracting target features, and matching the extracted same target features to detect a sheltered target, obtain the motion state of the sheltered target, and calculating the motion state of the sheltered target and the motion state of a vehicle so as to determine whether the vehicle accelerates to pass or decelerates to avoid or not.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of blind area detection, in particular to a blind area detection method, a vehicle, a readable storage medium and a computer program product. BACKGROUND

[0002] A safe driver assistant system (SDAS) can collect environmental parameters inside and outside a vehicle by using monitoring devices (such as cameras and sensors) installed on the vehicle, identify, detect and track static and dynamic objects, and thus prompt the driver of possible dangers in necessary cases, so as to effectively reduce the probability of accidents and mitigate accident injuries. Generally, a deep learning model is used to process images collected by the monitoring devices, output position information and category information of a target object to be identified in the images, further check whether the output result meets a preset alarm strategy, and alarm when the alarm strategy is met.

[0003] In related technologies, the processing of occluded targets usually adopts a trajectory reservation and motion prediction manner to prevent missing identification of occluded targets from causing a collision with the vehicle. In this manner, the prediction of the position, speed, motion direction and trend of an invisible target is limited by the design of a filter, and residual data in a partially occluded state is not used, so that the motion change of the occluded target cannot be known, and thus problems of improper acceleration and braking often occur. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a blind area detection method, which detects a target and extracts a target feature, and then matches the extracted target feature to achieve detection of an occluded target, obtain a motion state of the occluded target, and calculate the motion state with a motion state of the ego vehicle to determine whether the ego vehicle accelerates through or decelerates to avoid.

[0005] The present application further provides a vehicle.

[0006] The present application further provides a readable storage medium.

[0007] The present application further provides a computer program product.

[0008] According to the blind area detection method of the first aspect of the present application, a pedestrian image is collected and pedestrians in the image are detected, point cloud data is collected when the pedestrians enter a blind area behind a large vehicle or a wall, joint features of the pedestrians in the pedestrian image are extracted as feature points, the point cloud data and the joint feature point data of the pedestrians are hierarchically fused and determined, if target fusion is successful, the point cloud data and the joint feature points of the pedestrians are matched, and the confidence of radar detection points is fused to realize tracking of the motion state of the pedestrians.

[0009] According to the blind area detection method, the target is detected and the target feature is extracted, the same target feature is matched, the occluded target is detected, the motion state is obtained, and the motion state is calculated with the motion state of the ego vehicle to determine whether the ego vehicle accelerates through or decelerates to avoid.

[0010] According to some embodiments of the present application, the blind area detection method further comprises: the vehicle controller estimates whether the ego vehicle and the target have a collision risk according to the position and attitude information of the occluded target, and decides to accelerate to overtake the target or decelerate to avoid the target according to the surrounding road conditions.

[0011] According to some embodiments of the present application, the method of collecting pedestrian images and detecting pedestrians in the images further comprises: a camera collects pedestrian images, detects pedestrians in the collected pedestrian images, and cuts out the positions in the images to stretch into a fixed-size ROI image.

[0012] According to some embodiments of the present application, the method of collecting pedestrian images by the camera further comprises: the camera uses a deep learning algorithm to collect pedestrian images.

[0013] According to some embodiments of the present application, the joint features of the pedestrian include arm joints, leg joints, and the motion state of the pedestrian is determined to be running or walking.

[0014] According to the blind area detection system of the second aspect of the present application, the system comprises: a camera for collecting pedestrian images; a radar for collecting point cloud data; an analysis module for analyzing the data obtained by the camera and the radar; and a vehicle controller for controlling the vehicle according to the analysis result of the analysis module.

[0015] According to the vehicle of the third aspect of the present application, the vehicle comprises a processor and a memory connected to the processor; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory to execute the blind area detection method.

[0016] According to the computer readable storage medium of the fourth aspect of the present application, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the blind area detection method.

[0017] According to the computer program product of the fifth aspect of the present application, the computer program is executed by the processor to implement the blind area detection method.

[0018] The beneficial effects of the embodiments of the present application are: Additional aspects and advantages of the present application will be apparent from the following description, taken in conjunction with the accompanying drawings, wherein: BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which: Fig. 1 is a structural schematic diagram of a blind area detection system according to an embodiment of the present application; Fig. 2 is a schematic diagram of a blind area detection method according to an embodiment of the present application.

[0020] REFERENCE NUMERALS 11, camera; 12, radar; 13, vehicle controller. DETAILED DESCRIPTION

[0021] Embodiments of the present application are described in detail below with reference to the accompanying drawings, the embodiments described with reference to the accompanying drawings are exemplary, and embodiments of the present application are described in detail below.

[0022] Reference is made below to Figs. 1-2 A blind area detection method according to an embodiment of the present application is described below, and the present application also proposes a blind area detection system, a vehicle, a computer readable storage medium and a computer program product.

[0023] Referring to the drawings, the blind area detection method of the embodiment of the present application comprises: Collecting pedestrian images and detecting pedestrians therein; Collecting point cloud data when the pedestrian enters the blind area behind the large vehicle or the wall; Further extracting joint features of the pedestrians in the pedestrian images as feature points; Pedestrian joint features play an important role in the field of computer vision and pattern recognition, especially in human pose estimation, action recognition and pedestrian re-identification tasks. These features usually involve the detection and analysis of human key points (also known as joints) to describe the posture and motion state of the human body.

[0024] Hierarchical fusion determination of the point cloud data and the joint feature point data of the pedestrian; If the target fusion is successful, the point cloud data and the joint feature points of the pedestrian are matched, and the confidence of the radar 12 detection points is fused to realize tracking of the motion state of the pedestrian.

[0025] The point cloud data is collected by the millimeter wave radar 12, which is a device that uses millimeter wave frequency bands (usually defined as 30GHz to 300GHz, corresponding to a wavelength of 1mm to 10mm) for detection and ranging. It works based on the principle of emission, reflection and reception of electromagnetic waves. Here is the basic working principle of the millimeter wave radar 12: Transmit signal: The millimeter wave radar 12 first generates a high-frequency millimeter wave signal, which is emitted outward through an antenna.

[0026] Encounter object reflection: When the emitted millimeter wave encounters an object in front, part of the energy is reflected back by the object. The intensity and phase of the reflection depend on the material, shape, and surface characteristics of the object.

[0027] Receive reflected signal: The receiving antenna in the radar 12 system is responsible for capturing these reflected millimeter wave signals. Due to the different distances of the objects, the time of arrival of the reflected waves at the receiver will also be different.

[0028] Calculate distance: According to the time difference between emission and reception of the echo, using the known speed of electromagnetic wave propagation (the speed of light), the distance between the radar 12 and the object can be accurately calculated. This is because the time required for a round trip of electromagnetic waves is proportional to the distance of the target.

[0029] Get more information: In addition to basic distance measurement, information about the moving speed of the object (Doppler effect) can be obtained by analyzing the frequency change of the reflected wave, and the direction and angular position of the object can be determined by comparing the signal differences received by multiple receiving antennas.

[0030] The millimeter wave radar 12 has a wide range of applications in autonomous driving cars, unmanned obstacle avoidance, security monitoring, etc. due to its high resolution and ability to work effectively in bad weather conditions such as fog and smoke. In addition, it can also penetrate some non-metallic materials, making it very useful in some special application scenarios.

[0031] Therefore, by detecting the target and extracting the target features, and then matching the extracted same target features, the detection of the occluded target is achieved, and its motion state is calculated with the motion state of the ego vehicle to determine whether the ego vehicle should accelerate through or decelerate to avoid.

[0032] And the blind area detection method further comprises: the vehicle control unit 13 estimates whether there is a risk of collision between the vehicle and the target according to the position and posture information of the blocked target, and decides to accelerate to overtake the target or decelerate to avoid the target according to the surrounding road conditions. That is, the vehicle control unit 13 can estimate the specific position and motion trajectory of the target through the position and posture information of the blocked target, so as to estimate whether there is a risk of collision between the vehicle and the target, and then decide to accelerate to overtake the target or decelerate to avoid the target according to the surrounding road conditions.

[0033] For example, when the surrounding road conditions are poor, the vehicle can decelerate to avoid the target; or when the surrounding road conditions are good, the vehicle can accelerate to overtake the target.

[0034] The method for collecting pedestrian images and detecting pedestrians in the images further comprises: the camera 11 collects pedestrian images, detects pedestrians in the collected pedestrian images, and cuts out the positions in the images to stretch them into ROI (Region of Interest) images of a fixed size. That is, the pedestrian images can be collected by the camera 11, and the pedestrians in the collected pedestrian images can be detected, and the positions in the images can be cut out to stretch them into ROI images of a fixed size. The ROI image refers to analyzing a specific part of the image instead of the entire image, which can significantly reduce the amount of calculation, improve the processing speed, and for complex images, limiting the processing range can reduce the complexity of the problem, making some subsequent operations (such as edge detection, color analysis) more efficient.

[0035] Further, the method for collecting pedestrian images by the camera 11 further comprises: the camera 11 uses a deep learning algorithm to collect pedestrian images. Deep learning simulates the way the human brain processes information by building a multi-layer neural network model to solve complex problems. The basis of deep learning is a network structure composed of a large number of nodes (or neurons).

[0036] And the joint features of the pedestrian include arm joints, leg joints, and determining the motion state of the pedestrian as running or walking. In this way, by obtaining the arm joints and leg joints of the pedestrian, for the motion recognition task, analyzing the change pattern of the joint features helps to understand the motion of the pedestrian. For example, by tracking the movement trajectory of the key points over a period of time, the type of motion that the human is performing can be recognized, such as walking, running, jumping, etc.

[0037] The blind area detection system according to the second aspect of the present application comprises: a camera 11 for collecting pedestrian images; a radar 12 for collecting point cloud data; an analysis module for analyzing the data obtained by the camera 11 and the radar 12; and a vehicle control unit 13 for controlling the vehicle according to the analysis result of the analysis module.

[0038] According to the vehicle of the third aspect of the present application, the vehicle comprises a processor and a memory connected to the processor in communication; the memory stores computer-executable instructions; and the processor executes the computer-executable instructions stored in the memory to implement the blind area detection method.

[0039] The memory can include a mass storage that stores data or instructions. It can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory can include removable or non-removable (or fixed) media. Where appropriate, the memory can be internal or external to the integrated gateway disaster recovery device. In some embodiments, the memory is non-volatile solid-state memory.

[0040] The processor can generate operation control signals according to instruction opcodes and timing signals, and complete the control of fetching and executing instructions.

[0041] The memory can include read-only memory (ROM), random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the steps implementing the method for detecting defects of a diaphragm provided by the present application. It can be understood that the structural components shown in the embodiments of the present application do not constitute a specific limitation on the structure of the computer device.

[0042] According to the computer-readable storage medium of the fourth aspect of the present application, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the blind area detection method.

[0043] According to the computer program product of the fifth aspect of the present application, the computer program product comprises a computer program, and the computer program is executed by the processor to implement the blind area detection method.

[0044] In the description of the application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0045] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example.

[0046] Although embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A blind zone detection method, characterized in that, include: Acquire pedestrian images and detect pedestrians within them; Point cloud data is collected when a pedestrian enters the blind spot behind a large vehicle or wall. Next, the joint features of the pedestrians are extracted from the pedestrian images as feature points; Hierarchical fusion determination is performed between point cloud data and pedestrian joint feature point data; If target fusion is successful, the point cloud data is matched with pedestrian joint feature points, and the confidence of radar detection points is fused to achieve tracking of pedestrian movement status.

2. The blind zone detection method according to claim 1, characterized in that, Also includes: The vehicle controller estimates the risk of collision between the vehicle and the target based on the position and attitude information of the obscured target, and decides whether to accelerate to overtake the target or decelerate to avoid the target based on the surrounding road conditions.

3. The blind zone detection method according to claim 1, characterized in that, The method for acquiring pedestrian images and detecting pedestrians therein further includes: The camera captures pedestrian images, detects pedestrians within the captured images, and crops and stretches the locations of pedestrians in the images into a fixed-size Region of Interest (ROI) image.

4. The blind zone detection method according to claim 3, characterized in that, The method for capturing pedestrian images using a camera also includes: The camera uses deep learning algorithms to capture images of pedestrians.

5. The blind zone detection method according to claim 1, characterized in that, The joint features of the pedestrian include: arm joints, leg joints, and the determination of whether the pedestrian is running or walking.

6. A blind spot detection system, characterized in that, include: Cameras are used to capture images of pedestrians; Radar is used to collect point cloud data; The analysis module is used to analyze the data acquired by the camera and the radar; The vehicle controller controls the vehicle based on the analysis results from the analysis module.

7. A vehicle, characterized in that, The vehicle includes a processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory to perform the blind zone detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the blind zone detection method as described in any one of claims 1-5.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the blind zone detection method as described in any one of claims 1-5.

Images