Pedestrian courtesy detection method, device and terminal equipment
By setting up cameras on vehicles for intersection detection and pedestrian detection, the problem of difficulty in covering each intersection and high installation costs in the existing technology is solved, and effective monitoring of vehicle behavior and internal enterprise applications are achieved.
Patent Information
- Application Number
- CN202110315130.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-10-23
- Filing Date
- 2021-03-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-03-24
AI Technical Summary
The existing pedestrian courtesy inspection technology is difficult to cover every intersection, and the installation cost is high, making it difficult to apply within the enterprise.
Set the camera on the vehicle and collect video information in real time for intersection detection. If pedestrians are detected in the intersection area and the vehicle is not polite, illegal alarm operations are carried out, which reduces installation costs and realizes monitoring of vehicle behavior.
It realizes effective monitoring of vehicle behavior, reduces installation costs, solves the problem that existing technology is difficult to cover every intersection, and improves the application capabilities within the enterprise.
Smart Images

Figure CN112949559B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of software application technology, and in particular, relates to a pedestrian courtesy detection method, device and terminal equipment. Background Art
[0002] When driving, drivers should pay attention to giving way to pedestrians at intersections. If drivers do not give way to pedestrians, it is easy to cause traffic accidents and endanger personal and property safety. Therefore, transportation companies should strengthen supervision and education of drivers.
[0003] However, the current technology for pedestrian courtesy detection at intersections mainly involves installing fixed cameras near the sidewalk to capture passing vehicles and detect whether the driver's behavior is illegal. This solution is difficult to cover every intersection, has high installation costs, and is mainly aimed at traffic supervision departments, and is difficult to apply within enterprises. Summary of the invention
[0004] In view of this, the embodiments of the present application provide a pedestrian courtesy detection method, device and terminal equipment to solve the problems that the existing pedestrian courtesy detection technology at intersections is difficult to cover every intersection, has high installation costs, and is difficult to apply within the enterprise.
[0005] A first aspect of an embodiment of the present application provides a pedestrian courtesy detection method, comprising:
[0006] Performing real-time intersection detection on first video information collected by a first vehicle-mounted camera;
[0007] If an intersection area is detected in the first video information, performing intersection pedestrian detection on the first video information;
[0008] If it is detected that there are pedestrians in the intersection area within a first preset time period before the intersection area disappears, a violation alarm operation is performed;
[0009] Alternatively, if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within the second preset time period, the violation alarm operation is performed.
[0010] A second aspect of an embodiment of the present application provides a pedestrian courtesy detection device, comprising:
[0011] A first intersection module, used for performing real-time intersection detection on first video information collected by a first vehicle-mounted camera;
[0012] A first pedestrian module, configured to perform pedestrian detection at the intersection on the first video information if an intersection area is detected in the first video information;
[0013] A violation alarm module, configured to execute a violation alarm operation if it is detected that a pedestrian exists in the intersection area within a first preset time period before the intersection area disappears;
[0014] Alternatively, the violation alarm module is used to execute the violation alarm operation if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within a second preset time period.
[0015] A third aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0017] A fifth aspect of an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to implement the steps of the above method.
[0018] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0019] In the pedestrian courtesy detection method of the present application, a camera is set on a vehicle, and intersection detection is performed based on the first video information collected by the first camera. If an intersection area is detected in the first video information, pedestrian detection is performed on the first video information, and a judgment is made based on corresponding judgment logic whether the vehicle gives way to pedestrians. If the vehicle does not give way to pedestrians, a violation alarm operation is performed. By installing the camera on the vehicle, it is not necessary to install a camera at every intersection to monitor the vehicle behavior, which greatly reduces costs and solves the problems of existing intersection pedestrian courtesy detection technology that is difficult to cover every intersection, has high installation costs, and is difficult to apply within the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 Schematic diagram of the implementation process of the pedestrian courtesy detection method provided in the embodiment of the present application;
[0022] Figure 2is a schematic diagram of a pedestrian courtesy detection device provided in an embodiment of the present application;
[0023] Figure 3 is a schematic diagram of a terminal device provided in an embodiment of the present application;
[0024] Figure 4 It is a schematic diagram of pedestrian detection at an intersection provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0026] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.
[0027] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.
[0028] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0029] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0030] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0031] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0032] Embodiment 1:
[0033] The following is a description of a pedestrian courtesy detection method provided in Example 1 of the present application. Figure 1 The pedestrian courtesy detection method in the first embodiment of the present application includes:
[0034] Step S101, performing real-time intersection detection on first video information collected by a first vehicle-mounted camera;
[0035] In an embodiment of the present application, a first vehicle-mounted camera may be provided on the vehicle, and the first vehicle-mounted camera is used to collect first video information.
[0036] The pedestrian courtesy detection device performs real-time intersection detection on the first video information, and detects in real time whether an intersection area appears in the first video information.
[0037] Step S102: If an intersection area is detected in the first video information, pedestrian detection is performed on the first video information, and then step S103 or step S104 is executed;
[0038] If an intersection area is detected in the first video information, it means that the vehicle has already driven to the intersection, and the driver of the vehicle should give way to pedestrians in accordance with the provisions of traffic regulations.
[0039] At this time, the pedestrian yielding detection device can detect pedestrians at the intersection on the first video information, and determine whether there are pedestrians in the intersection area and whether the vehicle yields to pedestrians based on the detection result. If the vehicle does not yield to pedestrians, it means that the vehicle driver violates the rules.
[0040] There are many ways to judge whether the driver of the vehicle violates the rules. In the process of practical application, it can be selected according to actual needs. In this embodiment, the behavior of the driver of the vehicle can be evaluated through the content of step S103, and the behavior of the driver of the vehicle can also be evaluated through the content of step S104.
[0041] The algorithm for detecting intersections and the algorithm for detecting pedestrians at intersections may be the same target detection algorithm or different target detection algorithms, and the selection of specific algorithms may be determined according to actual needs. For example, in some embodiments, the Yolo algorithm may be selected as the algorithm for detecting intersections and / or the algorithm for detecting pedestrians at intersections; in other embodiments, other target detection algorithms, such as the bounding box algorithm, the RCNN algorithm, the SPP-net algorithm, etc., may also be selected as the algorithm for detecting intersections and / or the algorithm for detecting pedestrians at intersections.
[0042] Step S103: if it is detected that there are pedestrians in the intersection area within a first preset time period before the intersection area disappears, a violation alarm operation is performed;
[0043] In some possible implementations, real-time intersection detection may be continuously performed, and when it is detected that the intersection area disappears, it indicates that a vehicle is passing through the intersection area.
[0044] At this time, it can be determined whether a pedestrian is detected in the intersection area within the first preset time period before the intersection area disappears. The first preset time period can be set according to actual conditions, for example, it can be set from N1 seconds before the time point when the intersection disappears to the time point when the intersection disappears. For example, if the intersection disappears at time point T1, the first preset time period can be [T1-N1, T1], where N1 is a user-defined value greater than 0.
[0045] If pedestrians can be detected in the intersection area within the first preset time period before the intersection area disappears, it means that the vehicle did not give way to pedestrians and passed through the intersection directly, and the vehicle driver is judged to have violated the rules and a violation alarm operation is executed.
[0046] If no pedestrians are detected in the intersection area within the first preset time period before the intersection area disappears, it means that the vehicle gives way to pedestrians. The vehicle driver will start the vehicle to pass through the intersection only after the pedestrians pass the vehicle or leave the intersection area, giving way to pedestrians in accordance with traffic regulations.
[0047] Step S104: If pedestrians are detected in the intersection area and the vehicle speed is not 0 within the second preset time period, the violation alarm operation is executed.
[0048] Alternatively, in some other possible implementations, since traffic regulations stipulate that "when pedestrians are crossing a crosswalk, the vehicle must stop and give way", it can be determined based on the vehicle speed whether the driver gives way to pedestrians.
[0049] When a pedestrian is detected in the intersection area in the first video information, it means that the pedestrian is close to the vehicle and is passing through the intersection. At this time, according to fixed traffic regulations, the vehicle should stop and give way, so the vehicle speed can be detected.
[0050] If the vehicle speed is not 0 within the second preset time period, it means that the vehicle driver has not stopped to yield the right of way, and the vehicle driver is judged to have violated the rules, and a violation alarm operation is executed.
[0051] If the vehicle speed is 0 during the second preset time period, it means that the vehicle driver stops and gives way to pedestrians in accordance with traffic regulations.
[0052] The second preset time period can be set according to actual conditions. In some embodiments, the second preset time period can be set from N2 seconds before the time point when pedestrians are present in the intersection area to the time point when pedestrians are detected in the intersection area. For example, when pedestrians are detected in the intersection area at time point T2, the first preset time period can be [T2-N2, T2], where N2 is a user-defined value greater than 0.
[0053] The content of the violation alarm operation can be set according to the actual situation. In some possible implementations, the execution of the violation alarm operation includes:
[0054] A1. Upload the first video information within the third preset time period to the management and control platform.
[0055] The third preset time period can be set according to actual conditions. For example, the third preset time period can be the first preset time period or the second preset time period, or can be a time period including the first preset time period or the second preset time period.
[0056] When the violation alarm operation is performed, the first video information within the third preset time period can be uploaded to the management and control platform as evidence. The staff can view the first video information within the third preset time period on the management and control platform to review whether the driver's behavior is in violation of the rules, and can use the first video information within the third preset time period as a basis for punishing and educating the driver.
[0057] In some other possible implementations, the violation alarm operation may include:
[0058] B1. Setting the editing state of the first video information within the third preset time period to a non-editable state.
[0059] When the vehicle's network status is poor, the vehicle may not be able to upload the first video information within the third preset time period to the control platform in a timely manner. In order to prevent the vehicle driver from deleting or tampering with the first video information within the third preset time period, the editing status of the first video information within the third preset time period can be set to a non-editable status.
[0060] After the editing state of the first video information within the third preset time period is set to an uneditable state, the vehicle driver has no authority to delete or tamper with the first video information in the uneditable state, so that the evidence can be preserved.
[0061] The first video information within the third preset time period may or may not be uploaded. For example, when the network status is good, the first video information within the third preset time period may be uploaded to the management and control platform; when the network status is poor, the first video information within the third preset time period may not be uploaded to the management and control platform, and after waiting for the driving to end, the staff directly reads the first video information within the third preset time period in a non-editable state from the pedestrian courtesy detection device; or, the first video information within the third preset time period may not be set to the upload function, and after each driving, the staff directly reads the first video information within the third preset time period in a non-editable state from the pedestrian courtesy detection device.
[0062] In addition to the above-mentioned violation alarm operations, the vehicle driver can also be warned through sound and light alarms, buzzer alarms, speaker alarms, vibration alarms, etc.
[0063] In this embodiment, the violation alarm operations such as uploading videos and locking videos can interact with the control platform to save the evidence of the driver's violation, which is convenient for the staff to review and serve as a basis for punishing and educating the driver. At the same time, since the driver is only warned through the violation alarm operation in this embodiment, the vehicle deceleration and other behaviors will not be enforced, which avoids affecting the driver's driving operation, improves the safety of vehicle driving, and can also reduce the driver's resistance.
[0064] Furthermore, the method further comprises:
[0065] C1. Performing real-time intersection detection on second video information collected by a second vehicle-mounted camera, wherein the focal length of the second vehicle-mounted camera is greater than that of the first vehicle-mounted camera;
[0066] In some possible implementations, a second vehicle-mounted camera may be further provided on the vehicle, wherein the focal length of the second vehicle-mounted camera is greater than that of the first vehicle-mounted camera.
[0067] The first on-board camera has a short focal length and a short shooting distance, and is mainly used to detect whether vehicles give way to pedestrians at intersections and whether vehicle drivers violate regulations. The second on-board camera has a long focal length and a long shooting distance, and can detect intersections and pedestrians earlier, prompting vehicle drivers to give way to pedestrians.
[0068] Therefore, the pedestrian courtesy detection device can perform real-time intersection detection on the second video information collected by the second vehicle-mounted camera to determine whether the vehicle is about to reach the intersection area.
[0069] C2. If an intersection area is detected in the second video information, pedestrian detection at the intersection is performed on the second video information;
[0070] If the pedestrian courtesy detection device detects an intersection area in the second video information, it means that the vehicle is about to reach the intersection area. At this time, the second video information can be subjected to intersection pedestrian detection to detect whether there are pedestrians passing through the intersection area.
[0071] C3. If pedestrians are detected in the intersection area, a courtesy prompt operation is performed.
[0072] If the pedestrian yielding detection device detects the presence of pedestrians in the intersection area, it means that pedestrians are passing through the intersection area. At this time, the vehicle driver should yield to the pedestrians first, so the pedestrian yielding detection device can perform a yielding prompt operation.
[0073] The implementation method of the courtesy prompt operation can be set according to actual conditions. In some embodiments, the pedestrian courtesy detection device can prompt the vehicle driver to give way to pedestrians through sound and light prompts, buzzer prompts, speaker prompts, vibration prompts, etc.
[0074] When performing real-time intersection detection, the intersection detection can be performed by detecting intersection markers. The intersection markers can be selected according to actual conditions, for example, intersection markers such as zebra crossings and traffic lights can be selected for intersection detection.
[0075] In an embodiment of the present application, a zebra crossing can be selected as an intersection landmark, and intersection detection can be performed in the same manner as detecting the zebra crossing.
[0076] Correspondingly, the performing real-time intersection detection on the first video information collected by the first vehicle-mounted camera includes:
[0077] D1. Performing real-time zebra crossing detection on the first video information collected by the first vehicle-mounted camera;
[0078] When a zebra crossing is selected as a road intersection landmark, real-time zebra crossing detection may be performed on the first video information collected by the first vehicle-mounted camera.
[0079] D2. If a zebra crossing is detected in the first video information, it is determined that an intersection area is detected in the first video information and the area where the zebra crossing is located is used as the intersection area.
[0080] If a zebra crossing is detected in the first video information, it means that an intersection area is detected. In this case, the area where the zebra crossing is located can be used as the intersection area.
[0081] Correspondingly, the performing real-time intersection detection on the second video information collected by the second vehicle-mounted camera includes:
[0082] E1. Performing real-time zebra crossing detection on the second video information collected by the second vehicle-mounted camera;
[0083] When a zebra crossing is selected as a road intersection landmark, real-time zebra crossing detection may be performed on the second video information collected by the second vehicle-mounted camera.
[0084] E2. If a zebra crossing is detected in the second video information, it is determined that an intersection area is detected in the second video information and the area where the zebra crossing is located is used as the intersection area.
[0085] If a zebra crossing is detected in the second video information, it means that an intersection area is detected. In this case, the area where the zebra crossing is located can be used as the intersection area.
[0086] When detecting pedestrians at intersections, multiple pedestrians may be detected, some of which are in the intersection area and some are not. It is necessary to distinguish between the two types of pedestrians.
[0087] Therefore, the detecting pedestrians at the intersection on the first video information includes:
[0088] F1. Perform pedestrian detection on the first video information;
[0089] First, pedestrian detection is performed on the first video information, and all detected pedestrians are marked.
[0090] F2. If a pedestrian is detected in the first video information, calculate the ratio of the intersection of the pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame;
[0091] If a pedestrian is detected in the first video information, a pedestrian frame corresponding to the pedestrian is obtained, the ratio of the intersection of the pedestrian frame and the intersection area to the pedestrian frame is calculated, and whether the pedestrian is located in the intersection area is determined based on the ratio.
[0092] F3. If the ratio is greater than a preset threshold, it is determined that there are pedestrians in the intersection area.
[0093] If the ratio is greater than the preset threshold, it means that pedestrians are at the intersection and the vehicle driver should give way to pedestrians.
[0094] If the ratio is less than a preset threshold, it means that the pedestrian is not in the intersection area and the vehicle driver can pass through the intersection area directly.
[0095] The preset threshold value may be set according to actual conditions. For example, in some embodiments, the preset threshold value may be set to 15%.
[0096] by Figure 4 For example, the area within the zebra crossing frame is used as the intersection area. After detecting a pedestrian, obtain the pedestrian frame corresponding to the pedestrian. Determine the intersection of the zebra crossing frame and the pedestrian frame, that is, the overlapping area in the figure. Calculate the ratio of the overlapping area to the pedestrian frame, and determine whether the ratio is greater than a preset threshold. After calculation, if the ratio of the overlapping area to the pedestrian frame is greater than the preset threshold, it is determined that there is a pedestrian in the intersection area; or, after calculation, if the ratio of the overlapping area to the pedestrian frame is less than the preset threshold, it is determined that the pedestrian is not in the intersection area.
[0097] Further, performing intersection pedestrian detection on the second video information includes:
[0098] G1. Perform pedestrian detection on the second video information;
[0099] When performing pedestrian detection on the second video information, all detected pedestrians are marked.
[0100] G2. If a pedestrian is detected in the second video information, calculate the ratio of the intersection of the pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame;
[0101] If a pedestrian is detected in the second video information, a pedestrian frame corresponding to the pedestrian is obtained, the ratio of the intersection of the pedestrian frame and the intersection area to the pedestrian frame is calculated, and whether the pedestrian is located in the intersection area is determined based on the ratio.
[0102] G3. If the ratio is greater than a preset threshold, it is determined that there are pedestrians in the intersection area.
[0103] If the ratio is greater than a preset threshold, it means that pedestrians are at the intersection and vehicle drivers need to be reminded to give way to pedestrians.
[0104] If the ratio is less than the preset threshold, it means that the pedestrian is not in the intersection area and there is no need to prompt the vehicle driver to give way to the pedestrian.
[0105] In the pedestrian courtesy detection method provided in the first embodiment of the present invention, in the pedestrian courtesy detection method of the present application, a camera is set on the vehicle, and intersection detection is performed based on the first video information collected by the first camera. If an intersection area is detected in the first video information, pedestrian detection is performed on the first video information, and whether the vehicle gives way to pedestrians is judged according to corresponding judgment logic. If the vehicle does not give way to pedestrians, a violation alarm operation is performed. By installing the camera on the vehicle, it is not necessary to install a camera at every intersection to complete the monitoring of vehicle behavior, which greatly reduces costs and solves the problems of existing intersection pedestrian courtesy detection technology that is difficult to cover every intersection, has high installation costs, and is difficult to apply within the enterprise.
[0106] At the same time, in this embodiment, violation alarm operations such as uploading videos and locking videos can be adopted, and the control platform can be interacted with to save the evidence of the driver's violation, which is convenient for the staff to review and serve as the basis for punishing and educating the driver. At the same time, since in this embodiment, the driver is only warned through the violation alarm operation, the vehicle deceleration and other behaviors will not be enforced, which avoids affecting the driver's driving operation, improves the safety of vehicle driving, and can also reduce the driver's resistance.
[0107] In addition, a second on-board camera can be installed on the vehicle. The focal length of the second on-board camera is greater than that of the first on-board camera, so the second on-board camera can detect the intersection area and pedestrians earlier. By taking advantage of the difference in shooting distances between the first on-board camera and the second on-board camera, not only can the driver's violation be detected, but also a courtesy prompt operation can be performed before the driver violates the law, thereby reducing the driver's violation.
[0108] When performing intersection detection, the zebra crossing can be used as an intersection landmark, and the intersection area can be detected in the same way as the zebra crossing to accurately determine whether the vehicle has reached the intersection area.
[0109] When detecting pedestrians at intersections, it is possible to determine whether there are pedestrians in the intersection area based on the ratio of the intersection of the pedestrian frame corresponding to the detected pedestrian and the intersection area to the pedestrian frame, thereby accurately determining whether it is necessary to prompt the vehicle driver and whether the vehicle driver has violated the rules.
[0110] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0111] Embodiment 2:
[0112] Embodiment 2 of the present application provides a pedestrian courtesy detection device. For ease of description, only the parts related to the present application are shown, such as Figure 2 As shown, the pedestrian courtesy detection device includes:
[0113] The first intersection module 201 is used to perform real-time intersection detection on the first video information collected by the first vehicle-mounted camera;
[0114] A first pedestrian module 202, configured to perform pedestrian detection at the intersection on the first video information if an intersection area is detected in the first video information;
[0115] A violation alarm module 203, configured to execute a violation alarm operation if it is detected that a pedestrian exists in the intersection area within a first preset time period before the intersection area disappears;
[0116] Alternatively, the violation alarm module 203 is configured to execute the violation alarm operation if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within a second preset time period.
[0117] Furthermore, the violation alarm module 203 includes:
[0118] The information uploading submodule is used to upload the first video information within the third preset time period to the management and control platform.
[0119] Furthermore, the device also includes:
[0120] A second intersection module, used for performing real-time intersection detection on second video information collected by a second vehicle-mounted camera, wherein the focal length of the second vehicle-mounted camera is greater than that of the first vehicle-mounted camera;
[0121] A second pedestrian module, configured to perform pedestrian detection at the intersection on the second video information if an intersection area is detected in the second video information;
[0122] The courtesy prompt module is used to perform a courtesy prompt operation if a pedestrian is detected in the intersection area.
[0123] Furthermore, the first intersection module 201 includes:
[0124] A first zebra crossing submodule, used for performing real-time zebra crossing detection on first video information collected by a first vehicle-mounted camera;
[0125] The first area submodule is used to determine that a road intersection area is detected in the first video information and use the area where the zebra crossing is located as the road intersection area if a zebra crossing is detected in the first video information.
[0126] Furthermore, the second intersection module includes:
[0127] A second zebra crossing submodule, used for performing real-time zebra crossing detection on the second video information collected by the second vehicle-mounted camera;
[0128] The second area submodule is used to determine that a road intersection area is detected in the second video information and use the area where the zebra crossing is located as the road intersection area if a zebra crossing is detected in the second video information.
[0129] Furthermore, the first pedestrian module 202 includes:
[0130] A first detection submodule, configured to perform pedestrian detection on the first video information;
[0131] A first ratio submodule, configured to calculate the ratio of the intersection of a pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame if a pedestrian is detected in the first video information;
[0132] The first determination submodule is used to determine that there are pedestrians in the intersection area if the ratio is greater than a preset threshold.
[0133] Furthermore, the second pedestrian module includes:
[0134] A second detection submodule, used for performing pedestrian detection on the second video information;
[0135] A second ratio submodule, for calculating the ratio of the intersection of a pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame if a pedestrian is detected in the second video information;
[0136] The second determination submodule is used to determine that there are pedestrians in the intersection area if the ratio is greater than a preset threshold.
[0137] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0138] Embodiment three:
[0139] Figure 3 Schematic diagram of the terminal device provided in the third embodiment of the present application. Figure 3 As shown, the terminal device 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, the steps in the above-mentioned pedestrian courtesy detection method embodiment are implemented, for example Figure 1Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 2 Functions of modules 201 to 203 are shown.
[0140] Exemplarily, the computer program 32 may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 32 in the terminal device 3. For example, the computer program 32 may be divided into a first intersection module, a first pedestrian module, and a violation alarm module, and the specific functions of each module are as follows:
[0141] A first intersection module, used for performing real-time intersection detection on first video information collected by a first vehicle-mounted camera;
[0142] A first pedestrian module, configured to perform pedestrian detection at the intersection on the first video information if an intersection area is detected in the first video information;
[0143] A violation alarm module, configured to execute a violation alarm operation if it is detected that a pedestrian exists in the intersection area within a first preset time period before the intersection area disappears;
[0144] Alternatively, the violation alarm module is used to execute the violation alarm operation if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within a second preset time period.
[0145] The terminal device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of the terminal device 3 and does not constitute a limitation on the terminal device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.
[0146] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0147] The memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 may also be an external storage device of the terminal device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the terminal device. The memory 31 may also be used to temporarily store data that has been output or is to be output.
[0148] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0149] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0150] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0151] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0154] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0155] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A pedestrian courtesy detection method, characterized in that: include: Performing real-time intersection detection on first video information collected by a first vehicle-mounted camera; If an intersection area is detected in the first video information, performing intersection pedestrian detection on the first video information; If it is detected that there are pedestrians in the intersection area within a first preset time period before the intersection area disappears, a violation alarm operation is performed; Alternatively, if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within the second preset time period, the violation alarm operation is performed; The execution of the violation alarm operation includes: setting the editing state of the first video information within the third preset time period to a non-editable state; uploading the first video information within the third preset time period to the management and control platform; The method further comprises: Performing real-time intersection detection on second video information collected by a second vehicle-mounted camera, wherein the focal length of the second vehicle-mounted camera is greater than that of the first vehicle-mounted camera; If an intersection area is detected in the second video information, performing intersection pedestrian detection on the second video information; If pedestrians are detected in the intersection area, a courtesy prompt operation is performed.
2. The pedestrian courtesy detection method according to claim 1, characterized in that: The real-time intersection detection of the first video information collected by the first vehicle-mounted camera includes: Performing real-time zebra crossing detection on first video information collected by a first vehicle-mounted camera; If a zebra crossing is detected in the first video information, it is determined that an intersection area is detected in the first video information and the area where the zebra crossing is located is used as the intersection area.
3. The pedestrian courtesy detection method according to claim 1, characterized in that: The real-time intersection detection of the second video information collected by the second vehicle-mounted camera includes: Performing real-time zebra crossing detection on the second video information collected by the second vehicle-mounted camera; If a zebra crossing is detected in the second video information, it is determined that an intersection area is detected in the second video information and the area where the zebra crossing is located is used as the intersection area.
4. The pedestrian courtesy detection method according to claim 1, characterized in that: The detecting pedestrians at the intersection based on the first video information includes: Performing pedestrian detection on the first video information; If a pedestrian is detected in the first video information, calculating the ratio of the intersection of the pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame; If the ratio is greater than a preset threshold, it is determined that there are pedestrians in the intersection area.
5. The pedestrian courtesy detection method according to claim 1, characterized in that: The detecting pedestrians at the intersection based on the second video information includes: Performing pedestrian detection on the second video information; If a pedestrian is detected in the second video information, calculating the ratio of the intersection of the pedestrian frame corresponding to the pedestrian and the intersection area to the pedestrian frame; If the ratio is greater than a preset threshold, it is determined that there are pedestrians in the intersection area.
6. A pedestrian courtesy detection device, characterized in that: include: A first intersection module, used for performing real-time intersection detection on first video information collected by a first vehicle-mounted camera; A first pedestrian module, configured to perform pedestrian detection at the intersection on the first video information if an intersection area is detected in the first video information; A violation alarm module, configured to execute a violation alarm operation if it is detected that a pedestrian exists in the intersection area within a first preset time period before the intersection area disappears; Alternatively, the violation alarm module is configured to execute the violation alarm operation if a pedestrian is detected in the intersection area and the vehicle speed is not 0 within a second preset time period; The execution of the violation alarm operation includes: The violation alarm module includes: an information uploading submodule, which is used to set the editing state of the first video information within the third preset time period to a non-editable state; and upload the first video information within the third preset time period to the management and control platform; The device also includes: A second intersection module, used for performing real-time intersection detection on second video information collected by a second vehicle-mounted camera, wherein the focal length of the second vehicle-mounted camera is greater than that of the first vehicle-mounted camera; A second pedestrian module, configured to perform pedestrian detection at the intersection on the second video information if an intersection area is detected in the second video information; The courtesy prompt module is used to perform a courtesy prompt operation if a pedestrian is detected in the intersection area.
7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
