Pedestrian distance calculation method, system, computer and readable storage medium

Abstract

The present invention discloses a method, system, computer and readable storage medium for calculating the distance between pedestrians. The method includes: obtaining an image frame captured by a camera and performing a dangerous area calibration; determining whether a pedestrian exists in the image frame; if so, determining whether the pedestrian is in the calibrated dangerous area; if the pedestrian is in the calibrated dangerous area, querying whether the pedestrian appears in the camera for the first time, and detecting whether the image information of the pedestrian in the image frame is truncated; if so, calibrating the category of the pedestrian; and calibrating the height of the pedestrian to obtain the height value of the pedestrian; simulating the contact point between the pedestrian's feet and the ground in the image frame based on the height value of the pedestrian; calculating the coordinate value of the contact point in the image frame, and calculating the distance from the pedestrian to the camera. The beneficial effect of the present application is that the distance between the pedestrian and the vehicle can be calculated when the pedestrian suddenly appears or at a relatively close distance, making the forward collision warning function of the vehicle more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a pedestrian distance calculation method, system, computer and readable storage medium. Background Art

[0002] With the rapid development of the automotive industry, more and more cars are paying more attention to the development of vehicle safety functions. Among them, the vehicle's monocular ADAS system has become one of the mainstream configurations of vehicles. The monocular ADAS system includes lane departure function, forward collision warning function, pedestrian collision avoidance warning function, etc., which is used to combine with the camera module on the vehicle to realize assisted driving and safe driving functions.

[0003] In existing technology, the forward collision warning and pedestrian collision avoidance warning functions of monocular ADAS systems work by using a forward-facing camera mounted on the windshield to collect road information around the vehicle. A visual machine learning algorithm then detects obstacles ahead, such as vehicles, pedestrians, and bicycles. The height of the obstacle is calculated using pinhole imaging. Pre-calibrated camera parameters are then used to calculate the distance from the obstacle to the vehicle. Combining the preceding and following frame data, the relative speed of the obstacle and the vehicle can be calculated. This allows the collision time between the vehicle and the obstacle to be deduced, enabling early warning signals to be issued, thus avoiding accidents.

[0004] To prevent camera dirt and damage from affecting the vehicle's safety functions, the forward-facing cameras of most vehicles are installed on the windshield. However, due to the vehicle body structure, the cameras cannot fully obtain all road information in front of the vehicle. In the images captured by the cameras of pedestrians who are close to the vehicle, the lower half of the pedestrian's image information is often blocked by the vehicle's hood. This makes it impossible for the camera to infer the height of the pedestrian from the captured image, and further cannot calculate the distance between the pedestrian image and the vehicle in the truncated state by deduction, resulting in misjudgment of the forward collision warning function. Summary of the Invention

[0005] Based on this, one purpose of the present invention is to propose a pedestrian distance calculation method, system, computer and readable storage medium, which can realize the distance calculation between pedestrians and vehicles when pedestrians suddenly appear or at a close distance, making the vehicle's forward collision warning function more accurate.

[0006] The pedestrian distance calculation method proposed in an embodiment of the present invention is applied to a vehicle's monocular ADAS system to assist in implementing a pedestrian collision avoidance warning function. The method includes the following steps:

[0007] Characterized in that the method comprises:

[0008] Obtain image frames captured by the vehicle's forward-facing camera;

[0009] Performing dangerous area calibration on the image frame;

[0010] Determining whether there is a pedestrian in the image frame;

[0011] If there is a pedestrian in the image frame, determining whether the pedestrian is within a calibrated danger zone;

[0012] If the pedestrian is within the calibrated danger zone, query whether the pedestrian appears in the camera for the first time, and detect whether the image information of the pedestrian in the image frame is truncated;

[0013] If the pedestrian appears in the camera for the first time and the image information of the pedestrian is detected to be truncated in the image frame, the pedestrian is categorized according to the image information of the pedestrian; and the height of the pedestrian is calibrated according to the height value set for the category to obtain the height value of the pedestrian;

[0014] Simulating the contact points between the pedestrian's feet and the ground in the image frame according to the pedestrian's height;

[0015] The coordinate value of the contact point in the image frame is calculated according to the pinhole imaging principle, and the distance from the pedestrian to the camera is calculated according to the coordinate value.

[0016] The pedestrian distance calculation method proposed in this invention uses a camera to determine whether pedestrian images captured by the camera are truncated. If the images are truncated and appear for the first time, the pedestrian's approximate height is estimated based on their category. The method then uses camera extrinsic parameters and pinhole imaging principles to determine the coordinates of the pedestrian's feet and calculate the distance from the pedestrian to the camera. This method effectively overcomes the difficulty of distance calculation in situations where pedestrians appear to be "ghosting," improves the coverage of pedestrian distance calculation, and makes distance calculation more accurate when pedestrians are close to vehicles.

[0017] In addition, the pedestrian distance calculation method provided by the present invention may also have the following additional technical features:

[0018] Furthermore, the step of performing dangerous area calibration on the image frame includes:

[0019] Obtain vehicle speed information and steering wheel angle information based on vehicle body CAN data;

[0020] Calculating a driving area of ​​the vehicle within 2 seconds based on the vehicle speed information and the steering wheel steering angle information;

[0021] According to the camera external parameters, the driving area is mapped into the current image frame to form a dangerous area.

[0022] Furthermore, the step of querying whether the pedestrian appears in the camera for the first time includes:

[0023] Perform source tracing for pedestrians appearing in any image frame and detect whether the pedestrian exists in the corresponding area of ​​the previous image frame;

[0024] If the pedestrian does not exist in the corresponding area of ​​the previous frame image, the pedestrian is marked with an ID;

[0025] If the pedestrian exists in the corresponding area of ​​the previous frame image, the ID tag information in the previous frame image is used to perform ID tracking on the pedestrian.

[0026] Furthermore, after the step of marking the pedestrian's ID, the method further includes:

[0027] detecting whether image information of the pedestrian in the image frame is truncated;

[0028] If not, the contact points between the pedestrian’s footsteps and the ground in the image frame are finely segmented;

[0029] Calculating the coordinates of the contact point in the image frame based on the pinhole imaging principle, and calculating the distance from the pedestrian to the camera based on the coordinates;

[0030] Calculate the height of the pedestrian based on the principle of similar triangles, and store the height in the pedestrian's ID tag;

[0031] The height values ​​of the pedestrian recorded in multiple consecutive image frames are averaged, and the averaged value is the height value of the pedestrian.

[0032] Furthermore, after the steps of querying whether the pedestrian appears in the camera for the first time and detecting whether the image information of the pedestrian in the image frame is truncated, the method further includes:

[0033] If the pedestrian does not appear in the camera for the first time and the image information of the pedestrian in the image frame is detected to be truncated, the height value of the pedestrian is obtained based on the information marked with the pedestrian ID;

[0034] Simulating the contact point between the pedestrian's feet and the ground in the image frame according to the pedestrian's height value;

[0035] The coordinate value of the contact point in the image frame is calculated according to the pinhole imaging principle, and the distance from the pedestrian to the camera is calculated according to the coordinate value.

[0036] The present invention also provides a pedestrian distance calculation system, the system comprising:

[0037] Image acquisition module: used to acquire image frames taken by the vehicle's forward-facing camera;

[0038] Dangerous area calibration module: used for calibrating the dangerous area of ​​the image frame;

[0039] Pedestrian detection module: used to determine whether there is a pedestrian in the image frame;

[0040] Pedestrian positioning module: used to determine whether the pedestrian is within the calibrated danger zone;

[0041] First appearance detection module: used to check whether the pedestrian appears in the camera for the first time;

[0042] A truncation detection module is used to detect whether the image information of the pedestrian in the image frame is truncated;

[0043] Height calibration query module: used to calibrate the pedestrian's category based on the pedestrian's image information; and calibrate the pedestrian's height based on the height value set for the category to obtain the pedestrian's height value;

[0044] A contact point determination module is used to simulate the contact point between the pedestrian's feet and the ground in the image frame according to the pedestrian's height value;

[0045] Distance calculation module: used to calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

[0046] Furthermore, the dangerous area calibration module specifically includes:

[0047] Vehicle information acquisition unit: used to obtain vehicle speed information and steering wheel angle information based on vehicle body CAN data;

[0048] A driving area calculation unit is used to calculate the driving area of ​​the vehicle within 2 seconds based on the vehicle speed information and the steering wheel steering angle information;

[0049] Dangerous area generation unit: used to map the driving area into the current image frame according to the camera external parameters to form a dangerous area.

[0050] Furthermore, the first occurrence detection module specifically includes:

[0051] Source tracing query unit: used to perform source tracing query on pedestrians appearing in any image frame, and detect whether the pedestrian exists in the corresponding area within the previous frame of image;

[0052] ID tagging unit: used for tagging the pedestrian’s ID;

[0053] ID tracking unit: used to perform ID tracking on the pedestrian using the ID annotation information in the previous frame image.

[0054] In addition, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the pedestrian distance calculation method described in the first aspect above is implemented.

[0055] In addition, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the pedestrian distance calculation method as described in the first aspect above is implemented.

[0056] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0058] Figure 1 This is a flow chart of the pedestrian distance calculation method proposed in the first embodiment of the present invention;

[0059] Figure 2 This is a flow chart of a method for demarcating dangerous areas on image frames in the first embodiment of the present invention;

[0060] Figure 3 This is a flowchart of querying whether a pedestrian appears in the camera for the first time in the first embodiment of the present invention;

[0061] Figure 4 This is a flow chart of the method for marking pedestrians with IDs in the first embodiment of the present invention;

[0062] Figure 5 This is a flow chart of a method for querying whether a pedestrian appears in a camera for the first time and detecting whether the image information of the pedestrian in an image frame is truncated in the first embodiment of the present invention;

[0063] Figure 6 2 is a schematic diagram of the structure of a pedestrian distance calculation system according to the second embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0065] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0066] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0067] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0068] In existing technology, the forward collision warning and pedestrian collision avoidance warning functions of monocular ADAS systems work by using a forward-facing camera mounted on the windshield to collect road information around the vehicle. A visual machine learning algorithm then detects obstacles ahead, such as vehicles, pedestrians, and bicycles. The height of the obstacle is calculated using pinhole imaging. Pre-calibrated camera parameters are then used to calculate the distance from the obstacle to the vehicle. Combining the preceding and following frame data, the relative speed of the obstacle and the vehicle can be calculated. This allows the collision time between the vehicle and the obstacle to be deduced, enabling early warning signals to be issued, thus avoiding accidents.

[0069] To prevent camera dirt and damage from affecting the vehicle's safety functions, the forward-facing cameras of most vehicles are installed on the windshield. However, due to the vehicle body structure, the cameras cannot fully obtain all road information in front of the vehicle. In the images captured by the cameras of pedestrians who are close to the vehicle, the lower half of the pedestrian's image information is often blocked by the vehicle's hood. This makes it impossible for the camera to infer the height of the pedestrian from the captured image, and further cannot calculate the distance between the pedestrian image and the vehicle in the truncated state by deduction, resulting in misjudgment of the forward collision warning function.

[0070] To this end, the present invention proposes a pedestrian distance calculation method to overcome the problems existing in the prior art.

[0071] See also Figure 1 , a pedestrian distance calculation method proposed in the first embodiment of the present invention, is applied to a vehicle's monocular ADAS system to assist in implementing a pedestrian collision avoidance warning function. Specifically, the pedestrian distance calculation method includes the following steps:

[0072] Step S11: Acquire an image frame captured by the vehicle's forward-facing camera.

[0073] In an embodiment of the present invention, the vehicle's forward-facing camera is fixed to the vehicle's front windshield to avoid camera dirt and damage when the camera is placed externally, so that the image information obtained by the camera is clearer and more stable, which is conducive to image analysis and processing.

[0074] Step S12: calibrate the dangerous area of ​​the image frame.

[0075] By way of example and not limitation, in an embodiment of the present invention, a dangerous area can be expressed by setting a specified range area, or the driving trend of the vehicle can be simulated and predicted by associating information such as the current driving speed of the vehicle and the steering wheel turning angle to determine the dangerous area, and then further calibrate it.

[0076] Step S13: Determine whether there is a pedestrian in the image frame.

[0077] By way of example and not limitation, in an embodiment of the present invention, distance measurement is performed based on the situation where a pedestrian suddenly appears in front of the vehicle. In an embodiment of the present invention, other obstacles besides pedestrians are also detected and measured simultaneously to achieve a comprehensive anti-collision warning function.

[0078] Step S14: If there is a pedestrian in the image frame, determine whether the pedestrian is in the calibrated danger zone.

[0079] Through the dangerous area calibrated in step S12, it is determined whether the pedestrian is in the dangerous area, and then whether the current pedestrian is at risk of being hit. The analysis and calculation of pedestrians in the non-collision area are effectively screened out, which effectively reduces the operating load of the system and improves the computing efficiency.

[0080] Step S15: If the pedestrian is in the calibrated danger zone, it is checked whether the pedestrian appears in the camera for the first time, and whether the image information of the pedestrian in the image frame is truncated.

[0081] Among them, the method of querying whether the pedestrian appears in the camera for the first time can be achieved through ID tracking technology, and detecting whether the image information of the pedestrian in the image frame is truncated is achieved by judging whether the pedestrian in the image frame appears completely on the image frame, thereby determining whether the image information of the pedestrian is truncated.

[0082] Step S16: If the pedestrian appears in the camera for the first time and the image information of the pedestrian is detected to be truncated in the image frame, the pedestrian is categorized according to the image information of the pedestrian; and the height of the pedestrian is calibrated according to the height value set for the category to obtain the height value of the pedestrian.

[0083] It should be noted that the method of classifying pedestrians based on their image information is to find the target with the most similar image information to the new person in the established image database, and define the pedestrian's category based on the category of this target. Different categories are assigned a height reference value. After the pedestrian is categorized, it generates the corresponding height information. For example, when the current pedestrian is identified as a child, and the age range matched from the image database based on the recognized appearance features is between 9 and 12 years old, the gender is male, and the height reference value for men in this age group is calibrated to 1.4m, then the height of the pedestrian is immediately calibrated and determined to be 1.4m.

[0084] Step S17: Simulate the contact points between the pedestrian's feet and the ground in the image frame according to the pedestrian's height.

[0085] Step S18: Calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

[0086] Specifically, after determining the pedestrian's height, the pedestrian's size in the image frame is filled in, and then the coordinates of the pedestrian's feet in the image frame are depicted. Then, using the pinhole imaging principle, the distance between the pedestrian and the vehicle can be calculated through the intersection of the Y axis of the point coordinates and the vehicle ranging line.

[0087] In summary, the pedestrian distance calculation method provided by the present invention uses the aforementioned method to estimate the pedestrian's approximate height based on whether the pedestrian image information captured by the camera is truncated. If the pedestrian image is truncated and appears for the first time, the pedestrian's approximate height is estimated based on the pedestrian's category. The method then uses camera extrinsic parameters and the pinhole imaging principle to obtain the coordinates of the pedestrian's feet and calculate the distance from the pedestrian to the camera. This method effectively overcomes the difficulty of pedestrian distance calculation in the presence of "ghostly" pedestrians, improves the coverage of pedestrian distance calculation, and makes pedestrian distance calculation more accurate when the pedestrian is close to a vehicle.

[0088] For further information, see Figure 2 , is a method for calibrating dangerous areas on image frames in the pedestrian distance calculation method proposed in the first embodiment of the present invention, the method comprising the following steps:

[0089] Step S21: Acquire vehicle speed information and steering wheel angle information of the vehicle according to vehicle body CAN data.

[0090] Step S22: Calculate the vehicle's driving area within 2 seconds based on the vehicle speed information and the steering wheel angle information.

[0091] For purposes of illustration and not limitation, calculating the driving area of ​​a vehicle within 2 seconds in the embodiment of the present invention is only a preferred technical solution, and the present invention does not impose any specific limitation on the definition of the driving area.

[0092] Step S23: Map the driving area to the current image frame according to the camera external parameters to form a dangerous area.

[0093] In an embodiment of the present invention, the danger zone is defined by obtaining vehicle speed information and steering wheel angle information from the vehicle CAN bus, thereby enabling effective and accurate prediction of the vehicle's driving trend, thereby improving the definition of collision risks for pedestrians within the area.

[0094] For further information, see Figure 3 In the pedestrian distance calculation method proposed in the first embodiment of the present invention, the steps of querying whether a pedestrian appears in the camera for the first time are as follows:

[0095] Step S31: perform a source search on a pedestrian appearing in any image frame to detect whether there is a pedestrian in the corresponding area of ​​the previous image frame.

[0096] Step S32: If there is no pedestrian in the corresponding area of ​​the previous frame image, the pedestrian is marked with an ID;

[0097] Step S33: If there is a pedestrian in the corresponding area of ​​the previous frame image, the ID tag information in the previous frame image is used to track the pedestrian's ID.

[0098] In an embodiment of the present invention, the image frames of pedestrians are associated by adopting the technical means of ID tagging to achieve the purpose of pedestrian information tracking, facilitate the distinction of whether the pedestrian suddenly breaks in (ghostly), and adopt a corresponding method to realize the height calculation of the pedestrian.

[0099] For further information, see Figure 4 In the first embodiment of the present invention, after the step of marking the pedestrian's ID, the method further includes:

[0100] Step S41: Detect whether the image information of the pedestrian in the image frame is truncated.

[0101] Step S42: If not, finely segment the contact points between the pedestrian's footsteps and the ground in the image frame.

[0102] Step S43: Calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

[0103] Step S44: Calculate the height of the pedestrian based on the principle of similar triangles, and store the height value in the pedestrian's ID tag.

[0104] Step S45: average the height values ​​of the pedestrian recorded in multiple consecutive image frames, and the averaged value is the height value of the pedestrian.

[0105] Through the above technical solution, multiple frames of images are used to estimate the pedestrian's height and average the image, making the calculated pedestrian's height value more accurate, avoiding the situation where a single method of calculating the height has excessive deviation, and making the calculated height value more valuable for reference.

[0106] For further information, see Figure 5 , in the first embodiment of the present invention, after the steps of querying whether the pedestrian appears in the camera for the first time and detecting whether the image information of the pedestrian in the image frame is truncated, further comprising:

[0107] Step S51: If the pedestrian does not appear in the camera for the first time and the image information of the pedestrian in the detection image frame is truncated, the height value of the pedestrian is obtained according to the information marked with the pedestrian ID.

[0108] Step S52: Simulate the contact points between the pedestrian's feet and the ground in the image frame according to the pedestrian's height.

[0109] Step S53: Calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

[0110] The above technical solution effectively achieves accurate distance determination between pedestrians and vehicles, even when the pedestrian image is partially obscured. This improves the coverage of pedestrian-vehicle distance calculation and ensures accuracy even when pedestrian image information is incomplete.

[0111] See also Figure 6 , is a pedestrian distance calculation system proposed in the second embodiment of the present invention. Specifically, the pedestrian distance calculation system includes:

[0112] Image acquisition module 61: used to acquire image frames captured by the vehicle's forward-facing camera.

[0113] Dangerous area calibration module 62: used for calibrating dangerous areas of image frames.

[0114] Pedestrian detection module 63: used to determine whether there is a pedestrian in the image frame.

[0115] Pedestrian positioning module 64: used to determine whether a pedestrian is within a calibrated danger zone.

[0116] First appearance detection module 65: used to query whether a pedestrian appears in the camera for the first time.

[0117] The truncation detection module 66 is used to detect whether the image information of the pedestrian in the image frame is truncated.

[0118] The height calibration query module 67 is used to calibrate the pedestrian's category based on the pedestrian's image information; and calibrate the pedestrian's height based on the height value set for the category to obtain the pedestrian's height value.

[0119] The contact point determination module 68 is used to simulate the contact point between the pedestrian's feet and the ground in the image frame according to the pedestrian's height.

[0120] Distance calculation module 69: used to calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

[0121] Furthermore, the dangerous area calibration module 62 specifically includes:

[0122] Vehicle information acquisition unit: used to obtain vehicle speed information and steering wheel steering angle information based on vehicle body CAN data.

[0123] Driving area calculation unit: used to calculate the vehicle's driving area within 2 seconds based on vehicle speed information and steering wheel steering angle information.

[0124] Dangerous area generation unit: used to map the driving area into the current image frame according to the camera external parameters to form a dangerous area.

[0125] Furthermore, the first occurrence detection module 65 specifically includes:

[0126] Source tracing query unit: used to perform source tracing query on pedestrians appearing in any image frame, and detect whether there are pedestrians in the corresponding area within the previous frame image.

[0127] ID labeling unit: used to label pedestrians with IDs.

[0128] ID tracking unit: used to track pedestrians using the ID annotation information in the previous frame image.

[0129] Furthermore, the system also includes:

[0130] Image segmentation unit: used to finely segment the contact points between pedestrian footsteps and the ground in the image frame.

[0131] Distance calculation unit: used to calculate the coordinate value of the contact point in the image frame based on the pinhole imaging principle, and calculate the distance from the pedestrian to the camera based on the coordinate value.

[0132] Height calculation unit: used to calculate the height of the pedestrian based on the principle of similar triangles and store the height value in the pedestrian's ID tag.

[0133] Equalization processing unit: used to average the height values ​​of pedestrians recorded in multiple consecutive image frames. The averaged value is the height value of the pedestrian.

[0134] Information acquisition unit: used to obtain the height value of the pedestrian according to the information marked by the pedestrian ID.

[0135] In summary, the pedestrian distance calculation system provided by the present invention, combined with the above-mentioned pedestrian distance calculation method,

[0136] Based on whether the pedestrian image captured by the camera is truncated, and if it is truncated and appears for the first time, the pedestrian's approximate height is estimated based on their category. Then, using camera extrinsic parameters and the pinhole imaging principle, the coordinates of the pedestrian's feet are obtained to calculate the distance from the pedestrian to the camera. This effectively overcomes the problem of "ghostly" pedestrians appearing in pedestrian distance calculations, improves the coverage of pedestrian distance calculations, and makes pedestrian distance calculations more accurate when they are close to vehicles.

[0137] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0138] In addition, combined Figure 1 The pedestrian distance calculation method described in the embodiment of the present application can be implemented by a computer device. The computer device may include a processor and a memory storing computer program instructions.

[0139] Specifically, the processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0140] The memory may include a large-capacity memory for data or instructions. By way of example, and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to the data processing device. In a specific embodiment, the memory is non-volatile memory. In a specific embodiment, the memory includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0141] The memory may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor.

[0142] The processor implements any one of the pedestrian distance calculation methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0143] The computer device may also include a communication interface and a bus, wherein the processor, memory, and communication interface are connected via the bus and communicate with each other.

[0144] The communication interface is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of the present application. The communication interface can also enable data communication with other components such as external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0145] A bus, which includes hardware, software, or both, connects components of a computer device. It includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of the above. Where appropriate, a bus may include one or more buses. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0146] The computer device can execute the pedestrian distance calculation method in the embodiment of the present application based on the acquired data information, thereby realizing the combination of Figure 1 Describes the pedestrian distance calculation method.

[0147] In addition, in conjunction with the pedestrian distance calculation method in the above embodiments, the present application embodiment may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the pedestrian distance calculation methods in the above embodiments is implemented.

[0148] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0149] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A pedestrian distance calculation method, applied to a vehicle's monocular ADAS system to assist in realizing a pedestrian collision avoidance warning function, characterized in that: The method comprises: Obtain image frames captured by the vehicle's forward-facing camera; Performing dangerous area calibration on the image frame; Determining whether there is a pedestrian in the image frame; If there is a pedestrian in the image frame, determining whether the pedestrian is within a calibrated danger zone; If the pedestrian is within the calibrated danger zone, query whether the pedestrian appears in the camera for the first time, and detect whether the image information of the pedestrian in the image frame is truncated; If the pedestrian appears in the camera for the first time and the image information of the pedestrian is detected to be truncated in the image frame, the pedestrian is categorized according to the image information of the pedestrian; and the height of the pedestrian is calibrated according to the height value set for the category to obtain the height value of the pedestrian; Simulating the contact points between the pedestrian's feet and the ground in the image frame according to the pedestrian's height; The coordinate value of the contact point in the image frame is calculated according to the pinhole imaging principle, and the distance from the pedestrian to the camera is calculated according to the coordinate value.

2. The pedestrian distance calculation method according to claim 1, characterized in that: The step of performing dangerous area calibration on the image frame comprises: Obtain vehicle speed information and steering wheel angle information based on vehicle body CAN data; Calculating a driving area of ​​the vehicle within 2 seconds based on the vehicle speed information and the steering wheel steering angle information; According to the camera external parameters, the driving area is mapped into the current image frame to form a dangerous area.

3. The pedestrian distance calculation method according to claim 1, characterized in that: The step of querying whether the pedestrian appears in the camera for the first time includes: Perform source tracing for pedestrians appearing in any image frame and detect whether the pedestrian exists in the corresponding area of ​​the previous image frame; If the pedestrian does not exist in the corresponding area of ​​the previous frame image, the pedestrian is marked with an ID; If the pedestrian exists in the corresponding area of ​​the previous frame image, the ID tag information in the previous frame image is used to perform ID tracking on the pedestrian.

4. The pedestrian distance calculation method according to claim 3, characterized in that: After the step of marking the pedestrian's ID, the method further includes: detecting whether image information of the pedestrian in the image frame is truncated; If not, the contact points between the pedestrian’s footsteps and the ground in the image frame are finely segmented; Calculating the coordinates of the contact point in the image frame based on the pinhole imaging principle, and calculating the distance from the pedestrian to the camera based on the coordinates; Calculate the height of the pedestrian based on the principle of similar triangles, and store the height in the pedestrian's ID tag; The height values ​​of the pedestrian recorded in multiple consecutive image frames are averaged, and the averaged value is the height value of the pedestrian.

5. The pedestrian distance calculation method according to claim 4, characterized in that: After the steps of querying whether the pedestrian appears in the camera for the first time and detecting whether the image information of the pedestrian in the image frame is truncated, the method further includes: If the pedestrian does not appear in the camera for the first time and the image information of the pedestrian in the image frame is detected to be truncated, the height value of the pedestrian is obtained based on the information marked with the pedestrian ID; Simulating the contact point between the pedestrian's feet and the ground in the image frame according to the pedestrian's height value; The coordinate value of the contact point in the image frame is calculated according to the pinhole imaging principle, and the distance from the pedestrian to the camera is calculated according to the coordinate value.

6. A pedestrian distance calculation system, characterized in that: The system comprises: Image acquisition module: used to acquire image frames taken by the vehicle's forward-facing camera; Dangerous area calibration module: used for calibrating the dangerous area of ​​the image frame; Pedestrian detection module: used to determine whether there is a pedestrian in the image frame; Pedestrian positioning module: used to determine whether the pedestrian is within the calibrated danger zone; First appearance detection module: used to check whether the pedestrian appears in the camera for the first time; A truncation detection module is used to detect whether the image information of the pedestrian in the image frame is truncated; Height calibration query module: used to calibrate the pedestrian's category based on the pedestrian's image information; and calibrate the pedestrian's height based on the height value set for the category to obtain the pedestrian's height value; A contact point determination module is used to simulate the contact point between the pedestrian's feet and the ground in the image frame according to the pedestrian's height value; Distance calculation module: used to calculate the coordinate value of the contact point in the image frame according to the pinhole imaging principle, and calculate the distance from the pedestrian to the camera according to the coordinate value.

7. The pedestrian distance calculation system according to claim 6, characterized in that: The hazardous area calibration module specifically includes: Vehicle information acquisition unit: used to obtain vehicle speed information and steering wheel angle information based on vehicle body CAN data; A driving area calculation unit is used to calculate the driving area of ​​the vehicle within 2 seconds based on the vehicle speed information and the steering wheel steering angle information; Dangerous area generation unit: used to map the driving area into the current image frame according to the camera external parameters to form a dangerous area.

8. The pedestrian distance calculation system according to claim 6, characterized in that: The first occurrence detection module specifically includes: Source tracing query unit: used to perform source tracing query on pedestrians appearing in any image frame, and detect whether the pedestrian exists in the corresponding area within the previous frame of image; ID tagging unit: used for tagging the pedestrian’s ID; ID tracking unit: used to perform ID tracking on the pedestrian using the ID annotation information in the previous frame image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the pedestrian distance calculation method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the pedestrian distance calculation method according to any one of claims 1 to 5 is implemented.

Images