Commercial vehicle lateral blind area early warning method, device, equipment and medium

Abstract

The invention provides an early warning method, device and equipment for a lateral blind area of a commercial vehicle and a medium. The method comprises the following steps: acquiring the speed of a commercial vehicle and image data acquired by a wide-angle camera and a fisheye camera; when the speed of the commercial vehicle is smaller than or equal to a preset threshold value and the image data comprises vulnerable road users, preprocessing the image data; image perception is carried out on the preprocessed image data, and the position and the state of the vulnerable road user are identified; motion state fusion is carried out according to the position and state of the vulnerable road user and the position and state of the commercial vehicle; and when it is predicted that the vulnerable road user collides with the commercial vehicle according to the motion state fusion result, early warning information is sent out. Under the low-speed scene, the early warning method is provided for the lateral blind area of the commercial vehicle, and the safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a commercial vehicle lateral blind spot warning method, device, equipment and medium. Background Art

[0002] Commercial vehicles have many blind spots due to their complex and large body structures. Improving the driver's sight by adding rearview mirrors (such as blind spot compensation mirrors, front lower mirrors, etc.) is not effective, and various accidents caused by commercial vehicle blind spots still occur.

[0003] Therefore, in order to reduce driving accidents caused by blind spots, it is necessary to realize the early warning function of blind spots. Summary of the invention

[0004] The embodiments of the present invention provide a commercial vehicle lateral blind spot warning method, device, equipment and medium. The commercial vehicle lateral blind spot warning method provides a warning method for the commercial vehicle lateral blind spot in a low-speed scenario to improve safety.

[0005] According to one aspect of the present invention, a lateral blind spot warning method for a commercial vehicle is provided, wherein the commercial vehicle comprises a wide-angle camera and a fisheye camera arranged on at least one side of the commercial vehicle, and the lateral blind spot warning method comprises:

[0006] Acquire the speed of the commercial vehicle and the image data acquired by the wide-angle camera and the fisheye camera;

[0007] When the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes a vulnerable road user, preprocessing the image data;

[0008] Performing image perception on the pre-processed image data to identify the position and status of the vulnerable road user;

[0009] Performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle;

[0010] When a collision between the vulnerable road user and the commercial vehicle is predicted according to the motion state fusion result, a warning message is issued.

[0011] Optionally, after acquiring the speed of the commercial vehicle and the image data acquired by the wide-angle camera and the fisheye camera, the method further includes:

[0012] When the speed of the commercial vehicle is greater than a preset threshold, and / or the image data does not include vulnerable road users, discarding the current image data;

[0013] The speed of the commercial vehicle and the image data obtained by the wide-angle camera and the fisheye camera are continuously obtained until the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes vulnerable road users, and then subsequent steps of the lateral blind spot warning method are executed.

[0014] Optionally, preprocessing the image data includes:

[0015] Analyzing whether the size and format of the image data meet the model requirements;

[0016] If so, the image data is subjected to channel conversion, normalization and characterization processing and then stored to meet the subsequent data re-injection requirements.

[0017] Optionally, performing image perception on the preprocessed image data to identify the position and status of the vulnerable road user includes:

[0018] The image data is processed based on a deep neural network model to determine the position coordinates, speed information and acceleration information of the vulnerable road user.

[0019] Optionally, performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle includes:

[0020] Convert the 2D coordinate points output by the deep neural network model into 3D point coordinates in the real world;

[0021] determining whether the perceived vulnerable road user is moving toward or away from the commercial vehicle;

[0022] The motion information of the commercial vehicle is integrated into the motion state of the vulnerable road user to predict whether a collision occurs between the vulnerable road user and the commercial vehicle.

[0023] Optionally, the warning information includes a light warning or a sound warning.

[0024] Optionally, the preset threshold is 30 km / h, and the vulnerable road users include pedestrians, cyclists, electric bicycle riders or motorcycle riders.

[0025] According to another aspect of the present invention, a lateral blind spot warning device for a commercial vehicle is provided, comprising:

[0026] An acquisition module, used to acquire the speed of the commercial vehicle, and image data acquired by a wide-angle camera and a fisheye camera;

[0027] a preprocessing module, configured to preprocess the image data when the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes a vulnerable road user;

[0028] An image perception module, used to perform image perception on the pre-processed image data to identify the position and status of the vulnerable road user;

[0029] A motion state fusion module, used for performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle;

[0030] The early warning module is used to issue an early warning message when a collision between the vulnerable road user and the commercial vehicle is predicted based on the motion state fusion result.

[0031] According to another aspect of the present invention, there is provided a lateral blind spot warning device for a commercial vehicle, comprising:

[0032] one or more processors;

[0033] A memory for storing one or more programs;

[0034] When one or more programs are executed by one or more processors, the one or more processors implement the commercial vehicle lateral blind spot warning method provided by any embodiment of the present invention.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the commercial vehicle lateral blind spot warning method provided by any embodiment of the present invention is implemented.

[0036] The commercial vehicle lateral blind spot warning method provided by the embodiment of the present invention first obtains the speed of the commercial vehicle, and the image data obtained by the wide-angle camera and the fisheye camera; when the speed of the commercial vehicle is less than or equal to the preset threshold, and the image data includes vulnerable road users, the image data is preprocessed; then the preprocessed image data is image sensed to identify the position and state of the vulnerable road user; motion state fusion is performed based on the position and state of the vulnerable road user and the position and state of the commercial vehicle; when a collision between the vulnerable road user and the commercial vehicle is predicted based on the motion state fusion result, a warning message is issued. The technical solution of the embodiment of the present invention provides a warning method for the lateral blind spot of commercial vehicles in low-speed scenarios to improve safety.

[0037] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution in this embodiment, the drawings required for use in the description of the embodiment will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a process flow of a commercial vehicle lateral blind spot warning method provided by an embodiment of the present invention;

[0040] Figure 2 A schematic structural diagram of a commercial vehicle provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a typical working condition provided by an embodiment of the present invention;

[0042] Figure 4 A schematic flow chart of another commercial vehicle lateral blind spot warning method provided by an embodiment of the present invention;

[0043] Figure 5 It is a structural block diagram of a lateral blind spot warning device for a commercial vehicle provided by an embodiment of the present invention;

[0044] Figure 6 It is a schematic structural diagram of a commercial vehicle lateral blind spot warning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] Figure 1 A schematic diagram of a process flow of a commercial vehicle lateral blind spot warning method provided by an embodiment of the present invention, wherein the commercial vehicle comprises a wide-angle camera and a fisheye camera arranged on at least one side of the commercial vehicle, Figure 1 , the lateral blind spot warning methods include:

[0048] S110: Obtain the speed of the commercial vehicle and image data obtained by the wide-angle camera and the fisheye camera.

[0049] The lateral blind spot prediction method provided in this embodiment is applicable to low-speed scenarios of commercial vehicles, where the vehicle speed can be directly obtained through a speed sensor set on the commercial vehicle. The field of view of the wide-angle camera is greater than 90°. In specific implementation, a wide-angle lens with a field of view of 150° can be used.

[0050] For example, a wide-angle camera and a fisheye camera are installed on one side of a commercial vehicle. Figure 2 A structural schematic diagram of a commercial vehicle provided by an embodiment of the present invention, wherein image data collected by a wide-angle camera and a fisheye camera can be used as initial data for subsequent blind spot warnings.

[0051] S120: When the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes vulnerable road users, pre-process the image data.

[0052] The preset threshold can be designed according to the actual situation. In this embodiment, the preset threshold is 30 km / h. Vulnerable Road Users (VRU) include pedestrians, cyclists, electric bicycle riders or motorcycle riders. To determine whether the image data includes vulnerable road users, an image processing algorithm can be directly used, such as identifying whether the image has a human outline. The specific implementation can be designed according to the actual situation.

[0053] Optionally, preprocess the image data, including:

[0054] Analyze whether the size and format of image data meet the model requirements;

[0055] If so, the image data is stored after channel conversion, normalization and characterization processing to meet the subsequent data re-injection needs.

[0056] In specific implementation, to achieve data preprocessing, you can set up a data analysis module, a data processing module, and a data storage module. The data analysis module analyzes whether the input data meets the model requirements, including size, format, etc. If it meets the requirements, the data processing module performs channel conversion, normalization, characterization, etc. on the input data; if it does not meet the requirements, the data can be discarded. The data storage module mainly stores data used for inference to meet the subsequent data re-injection requirements.

[0057] S130: Perform image perception on the preprocessed image data to identify the position and status of vulnerable road users.

[0058] Optionally, image perception is performed on the pre-processed image data to identify the location and status of vulnerable road users, including:

[0059] Image data is processed based on a deep neural network model to determine the location coordinates, speed information and acceleration information of vulnerable road users.

[0060] During specific implementation, a deep neural network model is used for image perception, and a deep neural network model (hereinafter referred to as the model) pruning module, a model conversion module, a model quantization module, and a model acceleration module can be set. The model pruning module mainly deletes the channels in the model parameters that are not sensitive to the features, and minimizes the number of model parameters to reduce the computing power requirements of the SOC. The model conversion module mainly converts the trained model parameters into a file format that can be recognized by the SOC tool chain, so that the trained model can be accelerated in reasoning on the specified SOC. The model quantization module mainly reduces the memory usage of the model parameters, such as converting float32 to float16. The model acceleration module mainly refers to using the computing power of the SOC to perform matrix operations on the model, accelerate the computing efficiency of the model, and ensure the real-time performance of the perception algorithm.

[0061] S140: Perform motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle.

[0062] Optionally, motion state fusion based on the position and status of vulnerable road users and the position and status of commercial vehicles, including:

[0063] Convert the 2D coordinate points output by the deep neural network model into 3D point coordinates in the real world;

[0064] Determining whether the perceived vulnerable road user is moving towards or away from the commercial vehicle;

[0065] The motion information of commercial vehicles is integrated into the motion status of vulnerable road users to predict whether a collision will occur between vulnerable road users and commercial vehicles.

[0066] In specific implementation, when state fusion is performed, it is mainly composed of a coordinate point conversion module, a target orientation judgment module, a target speed calculation module and a fusion decision module. The coordinate point conversion can convert the 2D coordinate points output by the model into 3D point coordinates in the real world, and align them with the subsequent fusion decision module in the coordinate system dimension. The target orientation judgment module can calculate whether the perceived target is facing the current commercial vehicle or driving away from the current commercial vehicle, so that the subsequent modules can make accurate decisions. The target speed calculation module mainly calculates the current target's motion speed and acceleration and other information through the target motion distance between multiple frames of images. The fusion decision module fuses the objects recognized by the fisheye camera and the wide-angle camera.

[0067] Furthermore, a coordinate system conversion state prediction module can be included: the commercial vehicle motion information is integrated into the VRU motion state to realize the mobile coordinate system prediction function. The input of this module is the motion trajectory prediction of the VRU in the current vehicle coordinate system and the motion state of the test vehicle. The motion state of the test vehicle is taken into account in the change of the coordinate system, and the motion trajectory prediction of the VRU is adjusted from the fixed vehicle coordinate system to the mobile coordinate system to realize dynamic prediction.

[0068] S150: When a collision between a vulnerable road user and a commercial vehicle is predicted based on the motion state fusion result, a warning message is issued.

[0069] Optionally, the warning information includes a light warning or a sound warning. For example, when there is a risk of collision, the indicator light may be set to flash, or a buzzer may be set, or a voice prompt or other warning information may be set.

[0070] For example, Figure 3 A typical working condition diagram provided by an embodiment of the present invention takes a cyclist as an example of a VRU, and calculates LPI (Last Point Information) through several analyses of typical turning collision scenes.

[0071] Specifically, refer to Figure 3 , the test vehicle and VRU are driving at the same time, assuming that a collision occurs 8s later, the collision point is as follows Figure 3 As shown in ①, since the VRU is traveling in a straight line, the position 8s ago is the position of the VRU shown in the figure (collision point - 8×v bicycle ); Since the test vehicle has a turning motion, the position of the test vehicle 8s ago is: collision point - (8×v vehicle +L+R×θ-R×sinθ). Then calculate the LPI point. Since the driver's reaction time is 1.4s, according to the kinematic formula and considering the minimum collision distance, we can get Among them, 15 is the distance between the minimum LPI and the collision point, and the FPI (First Point Information) is the driving distance of 4s from the LPI. Based on the analysis of typical working conditions, a general warning strategy is obtained.

[0072] In other scenarios, the test vehicle may be stationary and the VRU may collide with the test vehicle from the front, rear or side. The lateral blind spot warning method provided in the embodiment of the present invention may also give a warning.

[0073] Figure 4 A flow chart of another commercial vehicle lateral blind spot warning method provided by an embodiment of the present invention, referring to Figure 4 , the lateral blind spot prediction method includes:

[0074] S210: Obtain the speed of the commercial vehicle and image data obtained by the wide-angle camera and the fisheye camera.

[0075] S220: When the speed of the commercial vehicle is greater than a preset threshold, and / or the image data does not include vulnerable road users, discard the current image data.

[0076] Among them, the commercial vehicle lateral blind spot prediction method provided in this embodiment is applicable to low-speed collision scenarios. When the speed of the commercial vehicle is greater than the preset threshold, the low-speed condition is not met. When the image data does not include vulnerable road users, there is no collision risk. In order to save storage space, the current image data is discarded in this case.

[0077] S230: Continue to obtain the speed of the commercial vehicle and the image data obtained by the wide-angle camera and the fisheye camera until the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes vulnerable road users, and then pre-process the image data.

[0078] S240: Perform image perception on the preprocessed image data to identify the position and status of the vulnerable road user.

[0079] S250: Perform motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle.

[0080] S260: When a collision between the vulnerable road user and the commercial vehicle is predicted according to the motion state fusion result, a warning message is issued.

[0081] S230 to S260 are similar to those in the previous embodiment and will not be described in detail here.

[0082] According to the same inventive concept, an embodiment of the present invention provides a lateral blind spot warning device for a commercial vehicle, Figure 5is a structural block diagram of a commercial vehicle lateral blind spot warning device provided by an embodiment of the present invention, with reference to Figure 5 , the commercial vehicle lateral blind spot warning device includes:

[0083] Acquisition module 1, used to acquire the speed of the commercial vehicle, and image data acquired by the wide-angle camera and the fisheye camera;

[0084] A preprocessing module 2, configured to preprocess the image data when the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes a vulnerable road user;

[0085] An image perception module 3, used for performing image perception on the pre-processed image data and identifying the location and status of vulnerable road users;

[0086] A motion state fusion module 4, for performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle;

[0087] The warning module 5 is used to issue a warning message when a collision between a vulnerable road user and a commercial vehicle is predicted based on the motion state fusion result.

[0088] The commercial vehicle lateral blind spot warning device provided by the embodiment of the present invention can execute the commercial vehicle lateral blind spot warning method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0089] According to the same inventive concept, an embodiment of the present invention provides a commercial vehicle lateral blind spot warning device, including:

[0090] one or more processors;

[0091] A memory for storing one or more programs;

[0092] When one or more programs are executed by one or more processors, the one or more processors implement the commercial vehicle lateral blind spot warning method provided by any embodiment of the present invention.

[0093] According to the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the commercial vehicle lateral blind spot warning method provided by any embodiment of the present invention is implemented.

[0094] Figure 6: is a schematic diagram of the structure of the commercial vehicle lateral blind spot warning device provided by an embodiment of the present invention. The commercial vehicle lateral blind spot warning device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The commercial vehicle lateral blind spot warning device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0095] like Figure 6 As shown, the commercial vehicle lateral blind spot warning device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the commercial vehicle lateral blind spot warning device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0096] Multiple components in the commercial vehicle lateral blind spot warning device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the commercial vehicle lateral blind spot warning device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0097] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a commercial vehicle lateral blind spot warning method.

[0098] In some embodiments, the commercial vehicle lateral blind spot warning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the commercial vehicle lateral blind spot warning device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the commercial vehicle lateral blind spot warning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the commercial vehicle lateral blind spot warning method in any other appropriate manner (for example, by means of firmware).

[0099] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0101] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein may be implemented on a commercial vehicle lateral blind spot warning device, which has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which a user can provide input to the commercial vehicle lateral blind spot warning device. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0103] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0104] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0105] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0106] The above specific implementations do not constitute a limitation on the scope of protection of this invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of this invention shall be included in the scope of protection of this invention.

Claims

1. A commercial vehicle lateral blind spot warning method, characterized in that: The commercial vehicle includes a wide-angle camera and a fisheye camera disposed on at least one side of the commercial vehicle, and the lateral blind spot warning method includes: Acquire the speed of the commercial vehicle and the image data acquired by the wide-angle camera and the fisheye camera; When the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes a vulnerable road user, preprocessing the image data; Performing image perception on the pre-processed image data to identify the position and status of the vulnerable road user; Performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle; When a collision between the vulnerable road user and the commercial vehicle is predicted according to the motion state fusion result, a warning message is issued.

2. The commercial vehicle lateral blind spot warning method according to claim 1, characterized in that: After acquiring the speed of the commercial vehicle and the image data acquired by the wide-angle camera and the fisheye camera, the method further includes: When the speed of the commercial vehicle is greater than a preset threshold, and / or the image data does not include vulnerable road users, discarding the current image data; The speed of the commercial vehicle and the image data obtained by the wide-angle camera and the fisheye camera are continuously obtained until the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes vulnerable road users, and then subsequent steps of the lateral blind spot warning method are executed.

3. The commercial vehicle lateral blind spot warning method according to claim 1, characterized in that: Preprocessing the image data includes: Analyzing whether the size and format of the image data meet the model requirements; If so, the image data is subjected to channel conversion, normalization and characterization processing and then stored to meet the subsequent data re-injection requirements.

4. The commercial vehicle lateral blind spot warning method according to claim 1, characterized in that: Performing image perception on the pre-processed image data to identify the position and status of the vulnerable road user includes: The image data is processed based on a deep neural network model to determine the position coordinates, speed information and acceleration information of the vulnerable road user.

5. The commercial vehicle lateral blind spot warning method according to claim 4, characterized in that: Performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle, including: Convert the 2D coordinate points output by the deep neural network model into 3D point coordinates in the real world; determining whether the perceived vulnerable road user is moving toward or away from the commercial vehicle; The motion information of the commercial vehicle is integrated into the motion state of the vulnerable road user to predict whether a collision occurs between the vulnerable road user and the commercial vehicle.

6. The commercial vehicle lateral blind spot warning method according to claim 1, characterized in that: The warning information includes light warning or sound warning.

7. The commercial vehicle lateral blind spot warning method according to claim 1 or 2, characterized in that: The preset threshold is 30 km / h, and the vulnerable road users include pedestrians, cyclists, electric bicycle riders or motorcycle riders.

8. A lateral blind spot warning device for commercial vehicles, characterized in that: include: An acquisition module, used to acquire the speed of the commercial vehicle, and image data acquired by a wide-angle camera and a fisheye camera; a preprocessing module, configured to preprocess the image data when the speed of the commercial vehicle is less than or equal to a preset threshold and the image data includes a vulnerable road user; An image perception module, used to perform image perception on the pre-processed image data to identify the position and status of the vulnerable road user; A motion state fusion module, used for performing motion state fusion according to the position and state of the vulnerable road user and the position and state of the commercial vehicle; The early warning module is used to issue an early warning message when a collision between the vulnerable road user and the commercial vehicle is predicted based on the motion state fusion result.

9. A commercial vehicle lateral blind spot warning device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the commercial vehicle lateral blind spot warning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the commercial vehicle lateral blind spot warning method as claimed in any one of claims 1 to 7 is implemented.

Images