Smart Cockpit Integrated System

Abstract

Provides a smart cockpit integrated system. The system includes a surround view image capturing device (120) and a surround view image detection server (110). The surround view image capturing device is used to capture surround view images around the vehicle. The surround view image detection server is connected to the surround view image capturing device to receive the surround view images. The surround view image detection server includes a pedestrian detection module (111), an image analysis module (112), and a warning module (113). The pedestrian detection module detects a pedestrian in the surround view image using a pedestrian detection model and then generates a detection frame surrounding the pedestrian image in the surround view image. The image analysis module determines a detection point on the detection frame based on the relative position of the detection frame with respect to the image center and calculates the pedestrian distance between the detection point and the image center. The warning module determines whether the pedestrian distance is less than a warning distance and, if so, generates a warning signal.

Description

[Technical Field]

[0001] The present disclosure relates to a smart cockpit integration system, and more particularly to a smart cockpit integration system that can detect pedestrians and issue corresponding warnings. [Background technology]

[0002] Conventional vehicle surround view systems use lenses with narrow viewing angles, requiring multiple lenses to be installed in multiple directions around the vehicle to compensate for blind spots in the driver's field of view. Furthermore, for large vehicles and machinery used off public roads, the safety requirements for their working environments are no less than those for driving on public roads. However, because their working environments are specialized and the types and quantities of vehicles and machinery are fewer than those of general vehicles, there are few examples of smart cockpit integrated systems being developed for such vehicles and machinery and their corresponding working environments, making it difficult to meet the safety requirements for special locations. Summary of the Invention [Problem to be solved by the invention]

[0003] The present disclosure provides a smart cockpit integration system including a surround view video capture device and a surround view video sensing server. [Means for solving the problem]

[0004] The present disclosure provides a smart cockpit integration system including a surround view image capture device and a surround view image detection server. The surround view image capture device is used to capture a surround view image of the vehicle's surroundings, with the surround view image extending into the air from a center below the surround view image capture device. The surround view image detection server is connected to the surround view image capture device to receive the surround view image and includes a pedestrian detection module, an image analysis module, and a warning module. The pedestrian detection module is used to detect a pedestrian in the surround view image using a pedestrian detection model and then generate a detection frame surrounding the pedestrian image in the surround view image. The image analysis module is connected to the pedestrian detection module and is used to determine a detection point on the detection frame based on the relative position of the detection frame with respect to the image center and calculate the pedestrian distance between the detection point and the image center. The warning module is connected to the image analysis module and is used to determine whether the pedestrian distance is within a warning distance and, if so, generate a warning signal.

[0005] In one embodiment, the pedestrian image ranges from the upper body to the entire body of the pedestrian in any rotational orientation within the surround view image.

[0006] In one embodiment, the warning module is further adapted to determine whether the pedestrian image does not correspond to the pedestrian's entire body, and if so, generate a warning signal.

[0007] In one embodiment, the video analysis module performs a distortion correction process on the detection window coverage area before calculating the pedestrian distance.

[0008] In one embodiment, the warning distance includes a first warning distance and a second warning distance, and the warning signal includes a first warning signal and a second warning signal, and the warning module generates the first warning signal when it determines that the pedestrian distance is equal to or less than the first warning distance, and generates the second warning signal when it determines that the pedestrian distance is equal to or less than the second warning distance.

[0009] In one embodiment, the acquisition area of ​​the plurality of training surround-view images for training the pedestrian detection model includes at least one of a mine, a forestry field, a cargo dock, a construction site, a farm, and a warehouse.

[0010] In one embodiment, the plurality of training surround-view images for training the pedestrian detection model includes a plurality of augmented surround-view images that have undergone data augmentation using viewing angle transformation.

[0011] In one embodiment, the smart cockpit integration system further includes a display device connected to the surround view image detection server, and the surround view image detection server further includes an image conversion module connected to the warning module, for converting the surround view image into at least one flat image and transmitting it to the display device for display. [Brief explanation of the drawings]

[0012] For a more complete understanding of the embodiments and their advantages, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a schematic diagram of a smart cockpit integration system in one embodiment of the present disclosure. [Figure 2] 1 is a schematic diagram illustrating installation of a surround view image capturing device according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a schematic rear view of the installation of a surround view image capture device according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is a schematic diagram of a surround view image according to an embodiment of the present disclosure. [Figure 5] 1 is a flowchart of a pedestrian detection method according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a schematic diagram illustrating pedestrian detection in a surround view image according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a schematic diagram of data augmentation of a training surround-view video in an embodiment of the present disclosure. [Figure 8]FIG. 1 is a schematic diagram illustrating a display of a surround view video in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013]

[0030] The following detailed description of exemplary embodiments of the present disclosure will be provided. However, it will be understood that the exemplary embodiments provide many applicable concepts that can be implemented in a variety of specific contexts. The exemplary embodiments of the disclosure discussed are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

[0014] The term "connected" as used in this disclosure means a direct or indirect electrical or communication connection, and the terms "first" and "second" are used merely to distinguish between multiple identical or similar conceptual elements and do not refer to a specific sequential relationship between the multiple elements. Furthermore, when a specific person, object, or event is described in this disclosure, it should be understood that the term refers to the corresponding specific person, object, or event in the video, even if "video" is not specifically added after the term.

[0015] The present disclosure provides a smart cockpit integrated system to sense the surroundings of a vehicle or equipment in a special area, and based on that, determine whether pedestrians are nearby and generate warnings, thereby ensuring the safety of pedestrians around the vehicle.

[0016] 1 is a schematic diagram of a smart cockpit integration system 100 according to one embodiment of the present disclosure. As shown in FIG. 1, the smart cockpit integration system 100 includes a surround view video detection server 110, a surround view video shooting device 120, a warning device 130, and a display device 140. The surround view video detection server 110 is connected to the surround view video shooting device 120, the warning device 130, and the display device 140. The surround view video detection server 110 includes a pedestrian detection module 111, a video analysis module 112, a warning module 113, and a video conversion module 114. The pedestrian detection module 111 is connected to the video analysis module 112, the video analysis module 112 is connected to the warning module 113, and the warning module 113 is connected to the video conversion module 114. Data or signals can be transmitted between the modules, but this is not limited to transmission between the modules connected to each other. After the surround view image around the vehicle is captured by the surround view image detection server 110, it is sent to the surround view image detection server 110 for pedestrian detection and analysis, a corresponding warning is issued by the warning device 130, and the corresponding image is displayed by the display device 140.

[0017] In one embodiment, the surround view image capture device 120 may be a fisheye lens. When the fisheye lens is aimed directly downward, it can capture surround view images with a 360-degree hemispherical range, with the downward point as the center of the image, and at least a 180-degree hemispherical range extending from the rear to the bottom and forward. Therefore, compared to a typical image capture device, even a small number of devices (e.g., one) can capture surround view images with an extremely wide viewing angle. The surround view image capture device 120 can be mounted on a vehicle (e.g., a general-purpose vehicle, a large construction vehicle, an agricultural vehicle, etc.). The surround view image detection server 110, the warning device 130, and the display device 140 can be devices installed in the vehicle and / or devices installed in a monitoring room and remotely connected. Alternatively, the surround view image detection server 110 can be a small device installed in the vehicle together with the surround view image capture device 120; however, this disclosure does not impose any particular limitations on these.

[0018] Areas where the smart cockpit integrated system 100 can be used include, for example, mines, forestry farms, cargo docks, construction sites, farmland, warehouses, etc., and the smart cockpit integrated system 100 can be installed on vehicles and machinery in these areas, such as forklifts, cultivators, and excavators. This allows an operator to use the smart cockpit integrated system 100 to detect whether nearby pedestrians (e.g., field workers) have appeared around the vehicle or machinery and issue a warning accordingly. The installation of the surround view video camera 120 in the smart cockpit integrated system 100 will be described below using a vehicle as an example.

[0019] FIG. 2 is a schematic diagram of the installation of surround view image capture device 120 in one embodiment of the present disclosure. As shown in the side view schematic diagram of FIG. 2(a), surround view image capture device 120 is installed at the rear of vehicle 200 using a bracket. In one embodiment, vehicle 200 is a forklift. Referring to the top view schematic diagram of FIG. 2(b), the front of vehicle 200 is facing direction D, and the range of surround view images captured by surround view image capture device 120 around the vehicle is range A at angle a. If surround view image capture device 120 is installed, for example, from a high position on the vehicle (such as on the roof or sunroof) via an extension bracket and attached at a position extended a distance d1 behind vehicle 200, the range of surround view images captured by surround view image capture device 120 around the vehicle is range B at angle b. Ranges A and B are used to indicate only the corresponding range of angles a and b, and the farthest distance that the surround view image capturing device 120 can capture is not limited to ranges A and B, but depends on the farthest distance that the actual device can capture.

[0020] As can be seen, the mounting position of the surround view image capturing device 120 on the vehicle 200 and the distance from the vehicle body affect the angle and range of the image that can be captured. Typically, the driver's seat of the vehicle 200 is installed at the front of the vehicle body (i.e., on the vehicle's 200 side in direction D), allowing the driver to directly see the surrounding scenery ahead. However, if the vehicle body is large, an image capturing device must be mounted on the vehicle body to supplement the driver's field of view. For example, the mounting position of the surround view image capturing device 120 on the vehicle 200 can be determined based on the type of vehicle body and the range that needs to be detected. Furthermore, in one embodiment, if the windows or body of the vehicle are transparent or not shielded, even if the surround view image capturing device 120 is mounted on the rear of the vehicle 200, a wider field of view can be detected through the unshielded parts of the windows or body compared to a non-transparent vehicle body, and areas to the side or front of the driver can be included.

[0021] The mounting height of surround view image capture device 120 on a vehicle also affects its capture range. Figure 3 is a rear-view schematic diagram of the mounting of surround view image capture device 120 in one embodiment of the present disclosure. Referring to Figure 3, if surround view image capture device 120 is mounted above vehicle 200, the capture range of vehicle 200 is limited by the vehicle body itself. For example, if surround view image capture device 120 is mounted at a distance d2 above vehicle 200, only the scenery within angles c1 and c2 will be captured, and ranges C1 and C2 will be blind spots for surround view image capture device 120.

[0022] As shown in FIG. 3 , if pedestrians H1 and H2 are standing on ground G around vehicle 200, pedestrian H1 is standing within angle c2, so his entire body is captured by surround view image capture device 120. However, pedestrian H2 is standing on ground G corresponding to range C2, so surround view image capture device 120 can only capture the upper half of pedestrian H1's body within angle c2. However, in this case, pedestrian H2 is closer to vehicle 200, meaning the location where pedestrian H2 is standing is more dangerous than pedestrian H1. Therefore, if surround view image capture device 120 needs to capture surround view images and then perform pedestrian detection and warning based on the surround view images, it must also be able to detect the half-body image of pedestrian H2. As can be seen from this, the captureable range can be determined by adjusting the installation height of surround view image capture device 120. Therefore, surround view image capture device 120 can be mounted, for example, via an extension bracket, in a high position on the vehicle (e.g., on the roof or sunroof). For example, if the height of the vehicle is 190 cm, the surround view image shooting device 120 can be installed at a height of 190 cm to 300 cm.

[0023] FIG. 4 is a schematic diagram of a surround view image 400 according to one embodiment of the present disclosure, showing an unprocessed image captured by the surround view image capture device 120. In one embodiment, the surround view image capture device 120 is mounted above the vehicle 200 and captures images facing downward. The surround view image 400 extends from the ground G toward the sky, centered on an image center 410 below the surround view image capture device 120. When pedestrians H3, H4, and H5 are standing normally, their feet are facing toward the image center 410. When observing and analyzing the surround view image 400 after capture, pedestrians H3, H4, and H5 may be rotated in any direction. Furthermore, as mentioned above, pedestrian H5 is closer to the vehicle body and therefore in the blind spot of the surround view image capture device 120. Therefore, only the upper body of pedestrian H5 is captured.

[0024] 5 is a flowchart of a surround view image sensing method 500 according to an embodiment of the present disclosure. The operation of the smart cockpit integration system 100 will now be described with reference to FIGS.

[0025] First, in step S510, a surround view image of the surroundings of the vehicle is captured using the surround view image capturing device 120, and the surround view image extends from the ground toward the sky around the center of the image. Next, in step S520, the surround view image detection server 110 receives the surround view image. In step S530, the pedestrian detection module 111 detects a pedestrian in the surround view image using a pedestrian detection model and generates a detection frame surrounding the pedestrian image in the surround view image.

[0026] FIG. 6 is a schematic diagram illustrating pedestrian detection in a surround view image 600 according to one embodiment of the present disclosure. In FIG. 6, because surround view image capture device 120 captures images facing directly downward, image center 410 of captured surround view image 600 is the ground directly below surround view image capture device 120, and the surrounding scenery is the scenery around surround view image capture device 120. As shown in FIG. 6, surround view image 600 includes pedestrians H6, H7, H8, and H9, which exist within surround view image 600 at different rotational directions around image center 410. Each pedestrian is surrounded by a detection frame F. Specifically, a pedestrian detection model is used to detect pedestrians within surround view image 600. When the pedestrian detection model detects the presence of a pedestrian within surround view image 600, it generates a corresponding detection frame F surrounding the pedestrian. Typically, this detection frame F is a rectangular frame that encloses the body area of ​​the detected pedestrian. For example, no matter what posture the pedestrian is in, what direction they are rotated in the surround view image, or whether they are the whole body or only the upper body (like pedestrian H9), if a pedestrian's presence is detected, they will be surrounded by this detection frame F. Here, the upper body refers to a range from not the whole body including the head to the whole body, and the present disclosure does not impose any particular limitations on this, as long as it can be detected by the pedestrian detection model and can be identified as a pedestrian, such as above the knees, above the waist, or above the chest.

[0027] The pedestrian detection model is an object detection model that can identify the presence of a pedestrian using supervised or unsupervised learning methods and can distinguish whether a detected pedestrian is half-body or full-body. When the surround view image 600 is captured using a fisheye lens, as shown in FIG. 4, the scene is distorted by the wide angle. Pedestrians H3, H4, and H5 are not oriented head-up and feet-down as in typical images, but rather appear in different rotational directions and may assume various postures, such as walking, standing, running, and crouching. The smart cockpit integrated system 100 is used in specialized areas other than general roads, such as mines, forestry farms, cargo docks, construction sites, farmland, and warehouses belonging to specific industries, and each area has its own challenges in pedestrian detection. For example, in forestry farms and farmland with tall crops, the tall and cluttered crops around the vehicle can easily obscure pedestrians or make them difficult to identify. For these reasons, the sources of training data required for the pedestrian detection model are fewer and more difficult than those for general roads. Therefore, a large amount of training surround-view footage for the pedestrian detection model can be captured in multiple special areas using a fisheye lens, and these training surround-view footage include footage of pedestrians in various movements, from at least the upper body to the whole body, and in any rotational direction.

[0028] In another embodiment, data augmentation may be performed on these training surround-view images to increase training data, for example, by partitioning and / or view angle conversion to obtain extended surround-view images. FIG. 7 is a schematic diagram illustrating data augmentation of training surround-view images in one embodiment of the present disclosure. As shown in FIG. 7 , training surround-view images 700 are images captured by a surround-view image capture device 120. By partitioning the original image into extended surround-view images 710 and 720, training data with different image sizes, resolutions, and orientations can be increased. Furthermore, since it is very important to train a pedestrian detection model to detect pedestrians with different rotational orientations, a pedestrian H10 included in the training surround-view image 700 can be rotated by view angle conversion to obtain a pedestrian H10' with a different rotational orientation, as in the extended surround-view image 720. After the image augmentation, a larger amount of images can be acquired to train the pedestrian detection model. Furthermore, as a data extension method, style conversion and processing for the degree of change in the image are also possible, and the present disclosure does not impose any particular limitations on these.

[0029] After the pedestrian image is surrounded in step S530, the process proceeds to step S540, where image analysis module 112 determines detection point S on detection frame F based on the relative position of detection frame F with respect to image center 410, as shown in Figure 6. Detection point S is determined by dividing surround view image 600 into a plurality of blocks based on image center 410, and dividing surround view image 600 into eight regions at 45-degree intervals by boundary lines L, for example (boundary lines L in Figure 6 are merely schematic and do not necessarily need to be drawn on surround view image 600). When the video analysis module 112 determines that the detection frame F of pedestrian H6 is in the area to the left of the image center 410, it determines the detection point S of pedestrian H6 to be on the right edge of the detection frame F; when it determines that the detection frame F of pedestrian H7 is in the area above the image center 410, it determines the detection point S of pedestrian H7 to be on the bottom edge of the detection frame F; and when it determines that the detection frame F of pedestrian H8 is in the area to the right and above the image center 410, it determines the detection point S of pedestrian H8 to be on the bottom left edge of the detection frame F. Similarly, when only the upper body of pedestrian H9 is captured, but it is determined that the detection frame F of pedestrian H9 is in the area to the lower right of the image center 410, it determines the detection point S of pedestrian H9 to be on the top left edge of the detection frame F.

[0030] In one embodiment, boundary line L can divide surround view image 600 into even more blocks, for example, dividing 360 degrees into blocks of one degree each, allowing for more precise determination of the position of detection point S. Alternatively, detection point S is defined as the point on which a line connecting the center of detection frame F and image center 410 passes through detection frame F, although the present disclosure does not impose any particular limitation thereon.

[0031] After determining detection point S, the process proceeds to step S550, where the image analysis module 112 calculates the pedestrian distance between detection point S and the image center 410, i.e., the distance between pedestrians H6 to H9 and the image center 410. The pedestrian distance can be calculated by converting image coordinates and spatial coordinates using the internal and external parameters of the camera to obtain the relative position of the pedestrian in space with the surround view image capture device 120 based on the pedestrian's position in the image. Furthermore, when calculating the pedestrian distance using the internal and external parameters of the camera, calibration, such as chessboard calibration, must be performed when installing the surround view image capture device 120 to obtain internal and external parameters such as focal length, coordinates, and distortion coefficients.

[0032] In one embodiment, when calculating the pedestrian distance, the video analysis module 112 performs distortion correction on the image area corresponding to the detection point S to obtain an undeformed planar image and calculate the pedestrian distance. Compared to a method of directly performing distortion correction after obtaining the surround view image 600 and then using a pedestrian detection model to detect, confirm the direction, and calculate the pedestrian distance, performing the above steps on the unprocessed surround view image 600, and then performing distortion correction on a partial image after confirming the detection point S, can reduce the computational costs required for the distortion correction process. In particular, the smart cockpit integrated system 100 of the present disclosure may be applied in an area with underdeveloped communications. In cases where cloud resources are unavailable and only local servers are available, or where only a small server is installed for convenience, reducing the computational complexity contributes to improved system performance and a wider range of applicable scenarios.

[0033] After calculating the pedestrian distance, the warning module 113 proceeds to step S560 to determine whether the pedestrian distance is equal to or less than the warning distance. If so, the warning module 113 proceeds to step S570 to generate a warning signal. If not, the warning module 113 repeats step S520, continues receiving surround view images, and performs subsequent operations such as pedestrian detection. However, different warning distances can be set as the criteria for generating a warning signal. For example, as shown in FIG. 6, different first and second warning distances R1 and R2 can be set (the first and second warning distances R1 and R2 in FIG. 6 are merely schematic and do not necessarily need to be depicted in the surround view image 600).

[0034] Specifically, in one embodiment, the first warning distance R1 may be set to a radius of 5 meters from the image center 410, and the second warning distance R2 may be set to a radius of 2 meters from the image center 410. As shown in FIG. 6, if the detection point S corresponding to pedestrian H6 is located outside the first warning distance R1, it indicates that the pedestrian is currently more than 5 meters away from the image center 410, and no corresponding warning signal is issued even if pedestrian H6 is detected. If the detection point S corresponding to pedestrian H7 is between the first warning distance R1 and the second warning distance R2, it indicates that the pedestrian is currently located between 2 and 5 meters from the image center 410, and therefore the warning module 113 determines and issues a corresponding first warning signal. If the detection point S corresponding to pedestrian H8 is located exactly at the first warning distance R1, it indicates that the pedestrian is currently located 5 meters away from the image center 410, and therefore the warning module 113 determines and issues a corresponding first warning signal. If the corresponding detection point S of pedestrian H9 is located within the second warning distance R2, it indicates that the pedestrian H9 is currently located within 2 meters from the image center 410, which means that the pedestrian H9 is very close to the vehicle, and therefore the warning module 113 judges and issues a corresponding second warning signal.

[0035] In one embodiment, in addition to determining the location of detection point S and the warning distance, the proportion of the detected pedestrian's body is also taken into consideration. For example, after pedestrian H9 is detected, if it is determined that the pedestrian's image does not include the entire body, for example, if only the upper body is detected, the lower half of the pedestrian's body is very likely to be in the blind spot of surround view image capture device 120, i.e., the position in range C2 where pedestrian H2 is standing in Figure 3. Standing in the blind spot means that the pedestrian is very close to the vehicle, so warning module 113 should issue a corresponding second warning signal.

[0036] In another embodiment, in addition to calculating the position of the detection point S corresponding to the pedestrian and determining whether to issue a corresponding warning based on the calculated position in steps S550 to S570, the surround view video detection server 110 can also receive the vehicle's current moving direction and speed from the vehicle system. After the pedestrian detection module 111 detects a pedestrian, the video analysis module 112 analyzes the pedestrian's predicted collision time based on the vehicle's current moving direction and speed, and the pedestrian's position, moving direction, and distance. If the pedestrian's predicted collision time is shorter than the collision time limit, the warning module 113 issues a corresponding warning signal.

[0037] After the warning module 113 issues a warning signal, the signal may be transmitted to a warning device 130, such as a speaker or warning light, mounted on the vehicle, or to a display device 140, which may prompt the driver inside the vehicle to stop and beware of surrounding pedestrians, alert surrounding pedestrians outside the vehicle, or transmit the signal to a vehicle or machine control room for a warning. The first and second warning signals may correspond to different presentation methods, for example, the first warning signal may correspond to a warning light emitting a continuous red light, while the more urgent second warning signal may correspond to a warning light emitting a red light and a speaker emitting a sound simultaneously. The warning device 130 may interact with the vehicle or machine control device to immediately stop or avoid the vehicle or machine to prevent an accident. The number of warning distances and corresponding warning signals may also be multiple, and the present disclosure is not limited thereto.

[0038] In one embodiment, to more intuitively convey the relationship between the pedestrian and the vehicle body and the corresponding warning to the driver, a simulation coordinate center is set at, for example, the center of the vehicle body or the position of the driver's seat, and the warning module 113 performs coordinate conversion of the pedestrian's distance and position based on the actual distance between the image center 410 and this simulation coordinate center. For example, if a pedestrian is located directly behind the vehicle body, 3 meters from the image center 410, and the actual distance between the image center 410 and the simulation coordinate center is 0.5 meters, the distance between the pedestrian and the simulation coordinate center is converted to 3.5 meters, and the corresponding spatial coordinates are calculated. In another embodiment, the position of the pedestrian on the boundary line L in FIG. 6 can be further converted to the pedestrian's position relative to the simulation coordinate center. Alternatively, in another embodiment, the image center 410 in FIG. 6 can be replaced with the simulation coordinate center, and the pedestrian's position and pedestrian distance can be obtained directly based on the relative position and distance between the simulation coordinate center and the detection frame.

[0039] As a result, the warning module 113 can determine the content of the warning signal based on the simulation coordinate center. For example, if a pedestrian is standing within 3 meters to the right of the vehicle center, the warning module 113 determines that the pedestrian is on the right side and issues a second warning signal indicating that the pedestrian is on the right side. If a pedestrian is standing 4 meters behind the vehicle center, the warning module 113 determines that the pedestrian is on the left rear side and issues a first warning signal indicating that the pedestrian is on the left rear side, and displays a schematic screen on the display device 140 or displays a corresponding position and colored light. For example, a schematic image of the vehicle may be used, and pedestrian images may be flashed and / or displayed in different colors or light symbols on the right and left rear sides of the vehicle image. In addition, the image analysis module 112 may improve the resolution of the directional divisions, allowing the warning signal to more accurately indicate the pedestrian's location.

[0040] In one embodiment, in consideration of the situation where the image of the surround view image capturing device 120 extends outward from the image center 410, and the captured object exists in different rotational directions and is partially distorted, if the actual image needs to be viewed on a monitor, such an image may be counterintuitive and difficult to recognize. Therefore, step S580 can be performed. After the surround view image detection server 110 receives the surround view image in step S520, the image conversion module 114 converts the surround view image into multiple flat images by image division and distortion correction, and then performs step S590 to transmit the converted images to the display device 140 for display, thereby enabling the driver or monitoring personnel to more easily monitor the surround view image.

[0041] 8 is a schematic diagram illustrating the display of a surround-view image in one embodiment of the present disclosure. As shown in FIG. 8, a planar image 810 represents the original 360-degree surround-view image unfolded. The image conversion module 114 can further divide the planar image 810 into a planar image 821 representing a 180-degree forward image, a planar image 822 representing a 90-degree left rear image, and a planar image 823 representing a 90-degree right rear image. The planar images 821, 822, and 823 are then displayed on the display device 140 to facilitate visual monitoring. However, since the surround-view image may lose data due to distortion correction and cropping during processing, steps S530 to S570, such as pedestrian detection, pedestrian distance calculation, and warning generation, are all performed on the original surround-view image.

[0042] According to the smart cockpit integrated system disclosed herein, a surround view imaging device is used to capture images of the area around a vehicle or machine in a special area, and based on the captured images, the system detects and determines whether a pedestrian is nearby and issues a warning, and further expands the data, thereby solving the problem of a lack of training data related to the special area.

[0043] The present disclosure has been disclosed in the examples as described above, but the above examples are not intended to limit the present disclosure, and a person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the scope of protection of the present disclosure is as defined by the scope of the utility model registration claims attached later. [Explanation of symbols]

[0044] 100:Smart Cockpit Integrated System 110: Surround view video detection server 111: Pedestrian detection module 112: Video analysis module 113: Warning module 114: Video conversion module 120: Surround view video recording device 130: Warning device 140:Display device 200: Vehicle 400, 600: Surround view video 410: Image focus 500: Surround view video detection method 700: Surround view training footage 710, 720: Extended surround view video 810, 821, 822, 823: Plane image A,B,C1,C2: Range a,b,c1,c2:Angle D: Direction d1,d2: distance F: Detection frame G: Ground H1~H10, H10': Pedestrians L: Boundary line R1: First warning distance R2: 2nd warning distance S: Detection point S510~S590: Process

Claims

1. A smart cockpit integration system, a surround view image capturing device used to capture a surround view image of the surroundings of a vehicle, the surround view image extending into the air with a center of the image below the surround view image capturing device; a surround-view image detection server connected to the surround-view image shooting device to receive the surround-view image; Equipped with The surround view video detection server a pedestrian detection module that detects the surround view image using a pedestrian detection model and then generates a detection frame surrounding the pedestrian image in the surround view image; an image analysis module connected to the pedestrian detection module, for determining a detection point on the detection frame based on a relative position of the detection frame with respect to the image center, and calculating a pedestrian distance between the detection point and the image center; a warning module connected to the video analysis module to determine whether the pedestrian distance is less than or equal to a warning distance, and if so, to generate a warning signal.

2. The smart cockpit integrated system according to claim 1 , wherein the pedestrian image ranges from the upper body to the entire body of the pedestrian in any rotation direction in the surround view image.

3. 3. The smart cockpit integration system of claim 2, wherein the warning module is further configured to determine whether the pedestrian image does not correspond to the entire body of the pedestrian, and if so, generate the warning signal.

4. The smart cockpit integration system of claim 1 , wherein the video analysis module performs distortion correction processing on the corresponding range of the detection frame before calculating the pedestrian distance.

5. 2. The smart cockpit integration system of claim 1, wherein the warning distance includes a first warning distance and a second warning distance, the warning signal includes a first warning signal and a second warning signal, and the warning module generates the first warning signal when it determines that the pedestrian distance is less than or equal to the first warning distance, and generates the second warning signal when it determines that the pedestrian distance is less than or equal to the second warning distance.

6. The smart cockpit integration system of claim 1 , wherein the acquisition area of ​​the plurality of training surround view images for training the pedestrian detection model includes at least one of a mine, a forestry field, a cargo dock, a construction site, farmland, and a warehouse.

7. The smart cockpit integration system of claim 1 , wherein the plurality of training surround view images for training the pedestrian detection model includes a plurality of extended surround view images that have undergone data extension through viewing angle transformation.

8. a display device connected to the surround-view video detection server, wherein the surround-view video detection server further includes a video conversion module connected to the warning module; The smart cockpit integrated system according to claim 1 , wherein the image conversion module is used to convert the surround view image into at least one flat image and transmit the flat image to the display device for display.

Images