Vehicle-mounted emergency interaction system and method

Abstract

The invention relates to the technical field of voice and pattern recognition, in particular to a vehicle-mounted emergency interaction system and method, and the system comprises an emergency release module and an emergency recognition module. According to the invention, by starting the emergency event publishing module, a driver describes the content and orientation of an emergency event in a language, and the emergency event publishing module converts the content information of the emergency event into a corresponding emergency event graphic symbol and displays the graphic symbol on the corresponding orientation of the car roof display screen according to the orientation information. The surrounding vehicles can identify the issued emergency graphic symbols by starting the emergency identification module to perform symbol escape, and then broadcast the content related to the emergency to the driver through the loudspeaker, so that the information interaction between the vehicles is realized, and the intention of the driver can be transmitted more quickly and more accurately, and the information of the emergency can be known.

Description

Technical Field

[0001] The present invention relates to the technical field of voice and image recognition, and more specifically, to a vehicle-mounted emergency event interaction system and method. Background Art

[0002] Currently, there is no mature technology for vehicles to interact with surrounding vehicles. In case of emergency, drivers generally use methods such as turning on hazard warning lights, frequently turning on and off high beams, opening windows and shouting to alert surrounding vehicles.

[0003] The information generated by traditional prompts is ambiguous, and surrounding vehicles cannot understand the driver's intentions, and cannot avoid a series of risks caused by this, such as the following scenarios: 1. Prompt that the rear of the vehicle in front is on fire; 2. Alert surrounding vehicles that there are patients in need of timely medical treatment, please give way; 3. Remind the vehicles behind to keep a safe distance; 4. Alert surrounding vehicles that the vehicle has a serious fault and asks them to avoid it; 5. Alert drivers of surrounding vehicles of sudden physical abnormalities and request assistance, etc.

[0004] Each of the above emergency events is closely related to driving safety. The existing vehicle-to-vehicle reminder method cannot meet the actual needs at all. Therefore, the present invention provides an in-vehicle emergency event interaction system and method to solve the above problems. Summary of the invention

[0005] The object of the present invention is to provide a vehicle-mounted emergency interaction system and method to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions: An in-vehicle emergency event interaction system includes an emergency event publishing module and an emergency event identification module; The emergency event release module and the emergency event identification module both include a central control unit, in which a speech recognition engine, an image recognition engine and a speech broadcast engine are arranged; The emergency event release module also includes an emergency event switch, a microphone and a display screen, wherein the emergency event switch, the microphone and the display screen are all connected to the central control unit, the microphone is used for the central control unit to obtain language related to the emergency event, the central control unit performs semantic analysis on the acquired language through a speech recognition engine and outputs corresponding emergency event graphic symbols, and the display screen is used to display the emergency event graphic symbols output by the central control unit; The emergency event recognition module also includes an image scanning switch, a speaker and a camera, and the image scanning switch, the speaker and the camera are all connected to the central control unit. The camera is used for the central control unit to obtain the emergency event graphic symbols displayed on the display screens of surrounding vehicles. The central control unit recognizes the emergency event graphic symbols through the image recognition engine and performs symbol escape to form corresponding emergency event text information. The central control unit synthesizes the emergency event text information into voice data through the voice broadcast engine and then broadcasts it through the speaker.

[0007] A preferred technical solution of the present application: the central control unit, emergency switch, microphone, image scanning switch and speaker are all arranged in the vehicle; The emergency switch and the image scanning switch are arranged on the steering wheel or the instrument panel. The emergency switch is used to control the opening and closing of the emergency event publishing module, and the image scanning switch is used to control the opening and closing of the emergency event identification module.

[0008] A preferred technical solution of the present application: the display screen and the camera are arranged on the roof, and the display screen and the camera are connected to the central control unit via a USB data cable; The display screen is a flexible display screen, the camera is a 360-degree rotating ultra-wide-angle camera, the display screen is bent into a columnar shape and installed on the roof, and the camera is installed on the top of the display screen.

[0009] A preferred technical solution of the present application: the microphone acquires language related to the emergency event including position information and event information, the position information includes the front, surroundings, rear, left and right, and the central control unit displays the emergency event graphic symbol on the display screen of the corresponding position according to the acquired position information, wherein when the position information is surroundings, the emergency event graphic symbol is displayed in all directions on the display screen.

[0010] A preferred technical solution of the present application is that the central control unit performs pixel partitioning on the columnar display screen, as follows: Front: the front half of the display screen 0°~180°; Rear: The rear half of the display screen 180°~360°; Left side: the left half of the display screen 90°~270°; Right side: the right half of the display screen 270°~90°; Surrounding: Full screen of the display 0°~360°; The central control unit maps the emergency graphic symbol into the pixel area of ​​the display screen at the corresponding position according to the position information.

[0011] A preferred technical solution of the present application is: when the emergency event recognition module is turned on, the camera rotates and scans the image information. When the emergency event graphic symbol displayed on the display screen of a distant vehicle is captured, the camera stops rotating and obtains the emergency event graphic symbol information.

[0012] A preferred technical solution of the present application: the image recognition engine of the central control unit uses opencv to perform image recognition.

[0013] The present application also provides a vehicle-mounted emergency event warning method, including an emergency event release strategy and an emergency event identification strategy; The emergency event release strategy includes the following steps: S11. Define a set of concrete emergency graphic symbols; S12, mapping the emergency graphic symbol defined in step S1 with the corresponding text description information and storing them in the central control unit to form a data file; S13, turning on the emergency switch, the driver speaks a language related to the emergency, and obtains language information through the microphone, the language information includes direction information and event information; S14, recognizing the language information obtained in step S3 through the speech recognition engine of the central control unit and performing semantic analysis, converting the position information into a corresponding position signal, matching the event information with the corresponding text description information in the data file, and mapping out the corresponding emergency event graphic symbol; S15, the central control unit displays the emergency event graphic symbol on the display screen of the corresponding position according to the acquired position signal, so as to realize the emergency event release; The emergency event identification strategy includes the following steps: S21. When the driver finds that an emergency event has been issued by a distant vehicle, he turns on the image scanning switch, and the camera on the roof rotates and scans the image information; S22, when capturing the emergency graphic symbol displayed on the display screen of the distant vehicle, the camera stops rotating and obtains the emergency graphic symbol information, and the central control unit recognizes the emergency graphic symbol information through the image recognition engine and performs symbol escape to form emergency text information; S23, the central control unit synthesizes the emergency event text information in step S22 into voice data through the voice broadcast engine, and then receives the voice data through the speaker to broadcast the content of the emergency event to the driver.

[0014] A preferred technical solution of the present application: In step S14, the specific recognition process of the image recognition engine of the central control unit is as follows: a. The camera continuously rotates to obtain image frames of the surrounding environment; b. Remove noise, grayscale and normalize the image; c. Use edge detection, shape feature, texture feature and color feature extraction methods to extract key features of emergency graphic symbols; d. Use data file matching or deep learning methods to identify emergency graphic symbols, determine the specific meaning of emergency graphic symbols, and obtain the description content of the emergency; e. According to the camera's shooting direction and the position of the emergency graphic symbol on the display screen, the actual direction of the emergency event is obtained.

[0015] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects: The present invention enables the emergency event publishing module, and the driver describes the content and location of the emergency event in words. The emergency event publishing module converts the content information of the emergency event into corresponding emergency event graphic symbols, and displays the graphic symbols on the corresponding location of the roof display screen according to the location information. Surrounding vehicles can identify the published emergency event graphic symbols by enabling the emergency event recognition module, perform symbol escape, and then broadcast the content related to the emergency event to the driver through the loudspeaker, thereby realizing information interaction between vehicles, and being able to convey the driver's intention faster and more accurately, and understand the information of the emergency event. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a system block diagram of the present invention; Figure 2 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is further described below in combination with the embodiments.

[0018] See also Figure 1~Figure 2 , an embodiment of the present application provides a vehicle-mounted emergency event interaction system, including an emergency event publishing module and an emergency event identification module; The emergency event release module and the emergency event identification module both include a central control unit, in which a speech recognition engine, an image recognition engine and a speech broadcast engine are arranged; The emergency event release module also includes an emergency event switch, a microphone and a display screen, wherein the emergency event switch, the microphone and the display screen are all connected to the central control unit, the microphone is used for the central control unit to obtain language related to the emergency event, the central control unit performs semantic analysis on the acquired language through a speech recognition engine and outputs corresponding emergency event graphic symbols, and the display screen is used to display the emergency event graphic symbols output by the central control unit; The emergency event recognition module also includes an image scanning switch, a speaker and a camera, and the image scanning switch, the speaker and the camera are all connected to the central control unit. The camera is used for the central control unit to obtain the emergency event graphic symbols displayed on the display screens of surrounding vehicles. The central control unit recognizes the emergency event graphic symbols through the image recognition engine and performs symbol escape to form corresponding emergency event text information. The central control unit synthesizes the emergency event text information into voice data through the voice broadcast engine and then broadcasts it through the speaker.

[0019] In the above-mentioned vehicle-mounted emergency interactive system, the central control unit, emergency switch, microphone, image scanning switch and speaker are all arranged in the vehicle; The emergency switch and the image scanning switch are arranged on the steering wheel or the instrument panel. The emergency switch is used to control the opening and closing of the emergency event publishing module, and the image scanning switch is used to control the opening and closing of the emergency event identification module.

[0020] Furthermore, the display screen and the camera are arranged on the roof of the vehicle, and the display screen and the camera are connected to the central control unit via a USB data cable; The display screen is a flexible display screen, the camera is a 360-degree rotating ultra-wide-angle camera, the display screen is bent into a columnar shape and installed on the roof, and the camera is installed on the top of the display screen. The camera can rotate 360 ​​degrees to capture images of the surrounding environment for identifying emergency graphic symbols on the display screens of surrounding vehicles; The language related to the emergency event acquired by the microphone includes position information and event information, the position information includes front, surrounding, rear, left and right, and the central control unit displays the emergency event graphic symbol on the display screen of the corresponding position according to the acquired position information, wherein when the position information is surrounding, the emergency event graphic symbol is displayed on the display screen in all directions; The central control unit performs pixel partitioning on the columnar display screen as follows: Front: the front half of the display screen 0°~180°; Rear: The rear half of the display screen 180°~360°; Left side: the left half of the display screen 90°~270°; Right side: the right half of the display screen 270°~90°; Surrounding: Full screen of the display 0°~360°; The central control unit maps the emergency event graphic symbol in the display screen pixel area of ​​the corresponding orientation according to the orientation information; A three-dimensional coordinate system is established with the center of the cylindrical display screen as the origin. The central axis of the cylindrical display screen is the Z axis, and the plane perpendicular to the axis is the XY plane. On the XY plane, the location of the emergency event is defined as the azimuth angle θ, which represents the angle of the emergency event relative to a fixed reference point on the display screen. According to the pixel area where the azimuth angle θ is located, the corresponding emergency event graphic symbol is displayed on the display screen in the area. The specific workflow is: 1. Driver operation: The driver presses the emergency switch; the driver describes the emergency content through the microphone, such as "reminding the vehicle ahead, the brake lights are not on."

[0021] 2. Voice recognition: The voice recognition engine recognizes the voice command: "Remind the vehicle ahead that the brake lights are not on."

[0022] 3. Generate control instructions: The central control unit generates a control command, the content is: Symbol type: Brake light off; Display position: front; Display direction: front 4. Transmission control instructions: The central control unit sends the control command to the display screen via the USB data cable; after receiving the control command, the display screen controls the circular flexible display screen to display the symbol of "brake light is off" in the front position.

[0023] Furthermore, when the emergency event recognition module is turned on, the camera rotates and scans the image information. When the emergency event graphic symbol displayed on the display screen of a distant vehicle is captured, the camera stops rotating and obtains the emergency event graphic symbol information. The image recognition engine of the central control unit uses OpenCV for image recognition, wherein OpenCV is a cross-platform computer vision and machine learning software library, and images can be recognized through OpenCV.

[0024] This embodiment provides a vehicle emergency warning method, including an emergency release strategy and an emergency identification strategy; The emergency event release strategy includes the following steps: S11. Define a set of concrete emergency graphic symbols; Among them, general emergency events include prompting that the rear of the vehicle in front is on fire; prompting surrounding vehicles that there are patients in the car who need timely medical treatment, please pay attention to avoid; prompting the rear vehicle to keep a safe distance; prompting surrounding vehicles that the car has a serious fault, please pay attention to avoid; prompting the drivers of surrounding vehicles that they have sudden physical abnormalities and request assistance; reminding the vehicle in front that the brake lights are not on, etc.; it can be determined according to actual conditions by defining a set of emergency symbols, each symbol represents an emergency, and the symbol corresponds to the event one by one. The symbol uses a vivid pattern to describe the emergency, which is conducive to the driver to intuitively feel the meaning of the symbol as soon as he sees it. The symbol is unique, which is conducive to the accurate recognition of the image recognition engine, thereby replacing the imperfection of the text description. For example, the brake light is not on, and the brake light pattern and the cross can be used to form a graphic symbol, and the red cross can be used to form a graphic symbol for requesting rescue. Other emergency events can be similarly composed of corresponding graphic symbols; S12, mapping the emergency graphic symbol defined in step S1 with the corresponding text description information and storing them in the central control unit to form a data file; S13, turning on the emergency switch, the driver speaks a language related to the emergency, and obtains language information through the microphone, the language information includes direction information and event information; S14, recognizing the language information obtained in step S3 through the speech recognition engine of the central control unit and performing semantic analysis, converting the position information into a corresponding position signal, matching the event information with the corresponding text description information in the data file, and mapping out the corresponding emergency event graphic symbol; In step S14, the specific recognition process of the image recognition engine of the central control unit is as follows: a. The camera continuously rotates to obtain image frames of the surrounding environment; Among them, the camera captures image frames at a certain frequency, such as 30 frames per second, and transmits these image frames to the central control unit; b. Remove noise, grayscale and normalize the image; The specific process of image processing is: Noise removal: Use methods such as median filtering or Gaussian filtering to remove noise from the image and improve image quality; Grayscale: Convert color images into grayscale images to reduce data volume and increase processing speed; Normalization: Normalize the image to keep its brightness and contrast consistent for subsequent processing; c. Use edge detection, shape feature, texture feature and color feature extraction methods to extract key features of emergency graphic symbols; The specific process of feature extraction is: Edge detection: Use the Canny edge detection algorithm or the Sobel operator to detect the edges in the image and extract the outline of the symbol; Shape features: Extract shape features of symbols, such as geometric characteristics of contours, area, perimeter, circularity, etc.; Texture features: Use gray-level co-occurrence matrix GLCM to extract the texture features of the symbol; Color features: extract the color features of symbols, such as color histogram; d. Use data file matching or deep learning methods to identify emergency graphic symbols, determine the specific meaning of emergency graphic symbols, and obtain the description content of the emergency; Among them, the deep learning method uses convolutional neural network (CNN) for symbol recognition. A CNN model is trained, with image features as input and symbol categories as output. Commonly used frameworks include TensorFlow, PyTorch, etc. Data preparation: Collect a large number of labeled symbol images for training models; Model training: Use the labeled data set to train the CNN model and optimize the model parameters; Model reasoning: Input new images into the trained model, and the model outputs the symbol category and its confidence; e. Obtain the actual direction of the emergency event based on the camera's shooting orientation and the position of the emergency event graphic symbol on the display screen; S15, the central control unit displays the emergency event graphic symbol on the display screen of the corresponding position according to the acquired position signal, so as to realize the emergency event release; The process of determining the position of emergency graphic symbols is as follows: Camera position information: The camera has a built-in gyroscope and accelerometer, which can obtain the camera's position information in real time. This information includes the camera's angle and direction; Image coordinate system conversion: convert the camera's orientation information into the orientation information in the image coordinate system. For example, when the camera is facing forward, the center point of the image corresponds to the front. Symbol position determination: Determine the actual direction of the symbol based on the symbol's position in the image and the camera's orientation information; for example, if the symbol appears on the left side of the image and the camera is facing forward, the actual direction of the symbol is on the left side; The emergency event identification strategy includes the following steps: S21. When the driver finds that an emergency event has been issued by a distant vehicle, he turns on the image scanning switch, and the camera on the roof rotates and scans the image information; S22, when capturing the emergency graphic symbol displayed on the display screen of the distant vehicle, the camera stops rotating and obtains the emergency graphic symbol information, and the central control unit recognizes the emergency graphic symbol information through the image recognition engine and performs symbol escape to form emergency text information; S23, the central control unit synthesizes the emergency event text information in step S22 into voice data through the voice broadcast engine, and then receives the voice data through the speaker to broadcast the content of the emergency event to the driver.

[0025] Through the above process, the emergency event release module can release the emergency event content described by the driver in the form of graphic symbols, and the image recognition module can accurately identify the emergency event graphic symbols of surrounding vehicles and inform the driver of relevant information in a timely manner to improve road driving safety.

[0026] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

[0027] In addition, it should be understood that although this specification is described according to implementation modes, not every implementation mode contains only independent technical solutions. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should take the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A vehicle-mounted emergency interactive system, characterized in that: It includes an emergency event release module and an emergency event identification module; The emergency event release module and the emergency event identification module both include a central control unit, in which a speech recognition engine, an image recognition engine and a speech broadcast engine are arranged; The emergency event release module also includes an emergency event switch, a microphone and a display screen, wherein the emergency event switch, the microphone and the display screen are all connected to the central control unit, the microphone is used for the central control unit to obtain language related to the emergency event, the central control unit performs semantic analysis on the acquired language through a speech recognition engine and outputs corresponding emergency event graphic symbols, and the display screen is used to display the emergency event graphic symbols output by the central control unit; The emergency event recognition module also includes an image scanning switch, a speaker and a camera, and the image scanning switch, the speaker and the camera are all connected to the central control unit. The camera is used for the central control unit to obtain the emergency event graphic symbols displayed on the display screens of surrounding vehicles. The central control unit recognizes the emergency event graphic symbols through the image recognition engine and performs symbol escape to form corresponding emergency event text information. The central control unit synthesizes the emergency event text information into voice data through the voice broadcast engine and then broadcasts it through the speaker.

2. The vehicle-mounted emergency interactive system according to claim 1, characterized in that: The central control unit, emergency switch, microphone, image scanning switch and speaker are all arranged in the vehicle; The emergency switch and the image scanning switch are arranged on the steering wheel or the instrument panel. The emergency switch is used to control the opening and closing of the emergency event publishing module, and the image scanning switch is used to control the opening and closing of the emergency event identification module.

3. The vehicle-mounted emergency interaction system according to claim 1 or 2, characterized in that: The display screen and the camera are arranged on the roof, and the display screen and the camera are connected to the central control unit via a USB data cable; The display screen is a flexible display screen, the camera is a 360-degree rotating ultra-wide-angle camera, the display screen is bent into a columnar shape and installed on the roof, and the camera is installed on the top of the display screen.

4. The vehicle-mounted emergency interaction system according to claim 3, characterized in that: The language related to the emergency event obtained by the microphone includes direction information and event information, and the direction information includes the front, surroundings, rear, left and right. The central control unit displays the emergency event graphic symbol on the display screen of the corresponding direction according to the obtained direction information, wherein when the direction information is surroundings, the emergency event graphic symbol is displayed in all directions on the display screen.

5. The vehicle-mounted emergency interaction system according to claim 4, characterized in that: The central control unit performs pixel partitioning on the columnar display screen as follows: Front: the front half of the display screen 0°~180°; Rear: The rear half of the display screen 180°~360°; Left side: the left half of the display screen 90°~270°; Right side: the right half of the display screen 270°~90°; Surrounding: Full screen of the display 0°~360°; The central control unit maps the emergency graphic symbol into the pixel area of ​​the display screen at the corresponding position according to the position information.

6. The vehicle-mounted emergency interactive system according to claim 3, characterized in that: When the emergency event recognition module is turned on, the camera rotates and scans the image information. When the emergency event graphic symbol displayed on the display screen of a distant vehicle is captured, the camera stops rotating and obtains the emergency event graphic symbol information.

7. The vehicle-mounted emergency interaction system according to claim 6, characterized in that: The image recognition engine of the central control unit uses opencv to perform image recognition.

8. A vehicle emergency warning method, characterized in that: Includes emergency release strategy and emergency identification strategy; The emergency event release strategy includes the following steps: S11. Define a set of concrete emergency graphic symbols; S12, mapping the emergency graphic symbol defined in step S1 with the corresponding text description information and storing them in the central control unit to form a data file; S13, turning on the emergency switch, the driver speaks a language related to the emergency, and obtains language information through the microphone, the language information includes direction information and event information; S14, recognizing the language information obtained in step S3 through the speech recognition engine of the central control unit and performing semantic analysis, converting the position information into a corresponding position signal, matching the event information with the corresponding text description information in the data file, and mapping out the corresponding emergency event graphic symbol; S15, the central control unit displays the emergency event graphic symbol on the display screen of the corresponding position according to the acquired position signal, so as to realize the emergency event release; The emergency event identification strategy includes the following steps: S21. When the driver finds that an emergency event has been issued by a distant vehicle, he turns on the image scanning switch, and the camera on the roof rotates and scans the image information; S22, when capturing the emergency graphic symbol displayed on the display screen of the distant vehicle, the camera stops rotating and obtains the emergency graphic symbol information, and the central control unit recognizes the emergency graphic symbol information through the image recognition engine and performs symbol escape to form emergency text information; S23, the central control unit synthesizes the emergency event text information in step S22 into voice data through the voice broadcast engine, and then receives the voice data through the speaker to broadcast the content of the emergency event to the driver.

9. The vehicle emergency warning method according to claim 8, characterized in that: In step S14, the specific recognition process of the image recognition engine of the central control unit is as follows: a. The camera continuously rotates to obtain image frames of the surrounding environment; b. Remove noise, grayscale and normalize the image; c. Use edge detection, shape feature, texture feature and color feature extraction methods to extract key features of emergency graphic symbols; d. Use data file matching or deep learning methods to identify emergency graphic symbols, determine the specific meaning of emergency graphic symbols, and obtain the description content of the emergency; e. According to the camera's shooting direction and the position of the emergency graphic symbol on the display screen, the actual direction of the emergency event is obtained.

Images