Data processing method in driving process, and related device, storage medium and program

Abstract

The application provides a data processing method for a safety hidden danger event in a driving process, and related devices, storage media and programs, which comprises the following steps: first, collecting image pictures of the driving process of a mixer truck through multiple cameras; determining that the running state of the mixing drum is rotation according to the image pictures; then, determining image segments in all directions centered on the mixing drum according to the image pictures; thereafter, judging whether a target direction image segment in the image segments in all directions exists a safety hidden danger event; if so, generating a warning information according to the safety hidden danger event; finally, outputting the warning information through a warning device to remind the driver of the mixer truck to pay attention to the safety hidden danger event. It can be seen that the method is beneficial to improving the intelligence and effectiveness of driving safety warning, and thus improving the driving safety.

Description

Technical Field

[0001] This application belongs to the general data processing technology field of the Internet industry, and specifically relates to a data processing method, related device, storage medium and program for safety hazard events during driving. Background Technology

[0002] During driving, drivers often encounter blind spots, making it difficult to accurately assess the road conditions around the concrete mixer truck and creating safety hazards. Moreover, compared to other vehicles, concrete mixer trucks are longer and have more blind spots, making it urgent to solve the problem of how to monitor the situation within the blind spots of the concrete mixer truck. Summary of the Invention

[0003] This application provides a data processing method and related products for safety hazard events during driving. The method aims to use the motion characteristics of the mixing drum through the vehicle body domain controller to process data on safety hazard events within the vehicle's blind spot, thereby ensuring the reliability of the corresponding warning information generation. This will help improve the intelligence and effectiveness of driving safety warnings and ultimately enhance driving safety.

[0004] In a first aspect, embodiments of this application provide a data processing method for safety hazard events during vehicle operation, applied to the body domain controller of a concrete mixer truck. The concrete mixer truck further includes a mixing drum and a warning device. Multiple cameras are mounted on the mixing drum, and the image acquisition range of the multiple cameras covers the entire angle centered on the mixing drum. The body domain controller is connected to the mixing drum, the multiple cameras, and the warning device, respectively. The method includes:

[0005] The multiple cameras capture video footage of the mixer truck during its operation.

[0006] When the operating state of the stirring drum is determined to be rotation based on the image, image segments in all directions centered on the stirring drum are determined based on the image.

[0007] Determine whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions, wherein the target direction includes the left and right sides of the mixer truck or the top of the mixer truck.

[0008] If such event exists, an early warning message is generated based on the aforementioned safety hazard event. The early warning message is used to indicate the existence of the aforementioned safety hazard event.

[0009] The warning information is output by the warning device to remind the driver of the mixer truck to pay attention to the potential safety hazard.

[0010] Secondly, embodiments of this application provide a data processing device for safety hazard events during vehicle operation, applied to the body domain controller of a concrete mixer truck. The concrete mixer truck further includes a mixing drum and a warning device. Multiple cameras are mounted on the mixing drum, and the image acquisition range of the multiple cameras covers the entire angle centered on the mixing drum. The body domain controller is connected to the mixing drum, the multiple cameras, and the warning device, respectively. The device includes:

[0011] The acquisition unit is used to acquire video images of the mixer truck driving process through the multiple cameras;

[0012] The determining unit is used to determine, based on the image frame, that the operating state of the stirring drum is rotation, and to determine image segments in all directions centered on the stirring drum based on the image frame.

[0013] The judgment unit is used to determine whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions, and the target direction includes the left and right sides of the mixer truck or the top of the mixer truck.

[0014] The early warning information generation unit is used to generate early warning information based on the safety hazard event if it exists, and the early warning information is used to indicate the existence of the safety hazard event;

[0015] The early warning unit is used to output the early warning information through the early warning device to remind the driver of the mixer truck to pay attention to the safety hazard.

[0016] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and one or more programs, the one or more programs being stored in the memory and configured to be executed by the processor, the programs including instructions for performing steps in the method as described in the first aspect of embodiments of this application.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program / instructions stored thereon, wherein the computer program / instructions, when executed by a processor, implement the steps of the method described in the first aspect of embodiments of this application.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described in the first aspect of embodiments of this application.

[0019] As can be seen from this embodiment, the vehicle body domain controller first acquires video images of the mixer truck's driving process through multiple cameras. Based on the video images, it determines that the mixing drum is rotating. Then, it identifies image segments in all directions centered on the mixing drum. Next, it determines whether there are any safety hazards in the target direction image segments among all the image segments. The target direction includes the left and right sides of the mixer truck or the top of the mixer truck. If a hazard exists, a warning message is generated based on the safety hazard. Finally, the warning message is output through a warning device to remind the mixer truck driver to pay attention to the safety hazard. Since the multiple cameras are mounted on the mixing drum, their image acquisition range covers all angles centered on the mixing drum. The video images of the mixer truck's driving process acquired by the vehicle body domain controller through these multiple cameras will completely include all image segments on the left and right sides of the mixer truck and the top of the truck. This allows the vehicle body domain controller to process data on safety hazards within the vehicle's blind spot based on the movement characteristics of the mixing drum, thereby ensuring the reliability of the generated warning message. This improves the intelligence and effectiveness of driving safety warnings, ultimately enhancing driving safety. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a structural block diagram of a mixer truck provided in an embodiment of this application;

[0022] Figure 2a This is a flowchart illustrating a data processing method for safety hazard events during driving, provided in an embodiment of this application.

[0023] Figure 2b This is a schematic diagram of a camera setup provided in an embodiment of this application;

[0024] Figure 2c This is a schematic diagram of another camera setup provided in an embodiment of this application;

[0025] Figure 2d This is a schematic diagram of an early warning page provided in an embodiment of this application;

[0026] Figure 3a This is a block diagram of the functional units of a data processing device for safety hazard events during driving, provided in an embodiment of this application.

[0027] Figure 3b This is another functional unit block of a data processing device for safety hazard events during driving provided in the embodiments of this application;

[0028] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0030] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] First, the system architecture involved in the embodiments of this application will be introduced.

[0033] Please see Figure 1 , Figure 1 This is a structural block diagram of a mixer truck provided in an embodiment of this application. For example... Figure 1As shown, the mixer truck 10 includes a vehicle body domain controller 11, a mixing drum 12, and a warning device 13. Multiple cameras 14 are mounted on the mixing drum 12, and the image acquisition range of the multiple cameras 14 covers the entire angle centered on the mixing drum 12. The vehicle body domain controller 11 is connected to the mixing drum 12, the multiple cameras 14, and the warning device 13. The vehicle body domain controller 11 can acquire video images of the mixer truck 10 during its operation through the multiple cameras 14, and then determine whether there are any safety hazards in the video images acquired by the multiple cameras 14. If so, it generates warning information based on the safety hazard event. This warning information is used to indicate the existence of a safety hazard event. Finally, the warning information is output through the warning device 13 to remind the driver of the safety hazard event.

[0034] Among them, the body domain controller 11 is a highly integrated and powerful domain controller that integrates all body electronics such as door locks, PEPS, wipers, interior and exterior lights, seats, tire pressure monitoring system (TPMS), and safety hazard event monitoring.

[0035] The warning device 13 may include, but is not limited to, a display device and / or a voice broadcasting device. The display device may be, but is not limited to, a display screen installed on the center console of the mixer truck and / or an external electronic device with a display screen connected to the vehicle domain controller. The voice broadcasting device may be, but is not limited to, an audio system installed on the center console of the mixer truck and / or an external buzzer connected to the vehicle domain controller. There are no specific limitations on the type or quantity of the warning device 13.

[0036] Based on this, the present application provides a data processing method for safety hazard events during driving. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0037] Please see Figure 2a , Figure 2a This is a flowchart illustrating a data processing method for safety hazard events during driving, provided in an embodiment of this application. The method is applied to, for example... Figure 1 The mixer truck 10 shown includes a vehicle domain controller 11. The mixer truck 10 also includes a mixing drum 12 and a warning device 13. Multiple cameras 14 are mounted on the mixing drum 12, and the image acquisition range of the multiple cameras 14 covers the entire angle centered on the mixing drum 12. The vehicle domain controller 11 is connected to the mixing drum 12, the multiple cameras 14, and the warning device 13. Figure 2a As shown, the method may include, but is not limited to, the following steps:

[0038] Step 201: Capture video footage of the mixer truck's driving process using the multiple cameras.

[0039] The aforementioned multiple cameras are arranged around the outer shell of the mixing drum. Each camera has a certain image acquisition range. The image acquisition ranges of different cameras may be the same or different. The key point is that the number of cameras is strongly correlated with the image acquisition range of the cameras, ensuring that the images captured by the multiple cameras cover the entire angle centered on the mixing drum, so as to ensure that the images captured by the multiple cameras include the left and right sides of the mixer truck and the images above the truck.

[0040] Specifically, the aforementioned multiple cameras can be installed on the outer casing of the stirring cylinder, which is perpendicular to the rotation center line of the stirring cylinder and has any cross-section within the aforementioned installation range.

[0041] For example, using three cameras—camera 1, camera 2, and camera 3—with a maximum image capture range of 120 degrees, please refer to [link / reference needed]. Figure 2b , Figure 2b This is a schematic diagram of a camera setup provided in an embodiment of this application, such as... Figure 2b As shown, cameras 1, 2, and 3 are installed on the outer shell of the mixing drum, perpendicular to its rotation center line and within the aforementioned cross-section A of the mixing drum. Any two cameras are equidistant from each other, ensuring that the image capture range of these three cameras covers the entire angle centered on the mixing drum. These three cameras capture images of the mixer truck's movement.

[0042] For another example, consider the four cameras used: camera 4 and camera 5 with a maximum image capture range of 120 degrees, and camera 6 and camera 7 with a maximum image capture range of 90 degrees. Please refer to [link to relevant documentation]. Figure 2c , Figure 2c This is a schematic diagram illustrating the setting range of another camera provided in an embodiment of this application, as shown below. Figure 2c As shown, cameras 4, 5, 6, and 7 are installed on the outer shell of the mixing drum, perpendicular to its rotation center line and within the aforementioned cross-section A of the mixing drum. This ensures that the image capture range of the four cameras covers the entire angle centered on the mixing drum. These four cameras capture images of the mixer truck's movement.

[0043] The camera settings described above are for illustrative purposes only and do not constitute a limitation on camera settings.

[0044] Step 202: Based on the image, determine that the stirring drum is rotating, and then determine image segments in all directions centered on the stirring drum based on the image.

[0045] The operating states of the stirring drum include rotation and stillness.

[0046] When the stirring drum is rotating, multiple cameras rotate with it, and the position of each camera changes constantly. The image frames corresponding to the image segments in the same direction centered on the stirring drum need to be obtained by cutting, splicing, and other operations on the image frames extracted from the images captured by the multiple cameras.

[0047] Step 203: Determine whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions.

[0048] The target direction includes the left and right sides of the mixer truck or the top of the mixer truck.

[0049] Step 204: If the safety hazard event exists, generate early warning information based on the safety hazard event. The early warning information is used to indicate the existence of the safety hazard event.

[0050] The warning information includes text prompts and / or video clips of the safety hazard event.

[0051] For example, in a pedestrian crossing scenario, if a safety hazard involves a pedestrian on the right side of a vehicle, the warning information could be a text message such as "Beware of pedestrians on the right side of the vehicle," and / or an image clip of the pedestrian on the right side of the vehicle.

[0052] For another example, in the scenario of driving in a height-restricted lane, if the safety hazard involves the height of the section of road where the cement mixer truck passes, the warning information could be a text message such as "Caution: Height-restricted lane, beware of collisions above vehicles," and / or an image clip of the area above the vehicle.

[0053] Step 205: Output the warning information through the warning device to remind the driver of the mixer truck to pay attention to the safety hazard.

[0054] As can be seen from this embodiment, the vehicle body domain controller first acquires video images of the mixer truck's driving process through multiple cameras. Based on the video images, it determines that the mixing drum is rotating. Then, it identifies image segments in all directions centered on the mixing drum. Next, it determines whether there are any safety hazards in the target direction image segments among all the image segments. The target direction includes the left and right sides of the mixer truck or the top of the mixer truck. If a hazard exists, a warning message is generated based on the safety hazard. Finally, the warning message is output through a warning device to remind the mixer truck driver to pay attention to the safety hazard. Since the multiple cameras are mounted on the mixing drum, their image acquisition range covers all angles centered on the mixing drum. The video images of the mixer truck's driving process acquired by the vehicle body domain controller through these multiple cameras will completely include all image segments on the left and right sides of the mixer truck and the top of the truck. This allows the vehicle body domain controller to process data on safety hazards within the vehicle's blind spot based on the movement characteristics of the mixing drum, thereby ensuring the reliability of the generated warning message. This improves the intelligence and effectiveness of driving safety warnings, ultimately enhancing driving safety.

[0055] For ease of understanding, the process of determining image segments in all directions centered on the stirring cylinder in the embodiments of this application will be described below.

[0056] In one possible example, determining image segments in all directions centered on the stirring tank based on the image frame may include, but is not limited to, the following steps:

[0057] Step A1: Determine continuous frame images with time axes in all directions centered on the stirring drum based on the image images.

[0058] Step A2: Generate image segments in all directions centered on the stirring drum based on the continuous frame images with time axes in all directions centered on the stirring drum.

[0059] Among them, various existing methods can be used to generate image segments centered on the stirring drum in that direction using continuous frame images with a time axis in one direction.

[0060] For example, multiple cameras, including camera 1, camera 2 and camera 3, can capture image 1, image 2, and image 3 respectively. Then, the image frames at the same time point in image 1, image 2 and image 3, which are in the same direction centered on the stirring drum, can be cut, spliced ​​and other operations can be performed to obtain the image frame corresponding to that direction at that time point.

[0061] As can be seen, in this example, the vehicle domain controller can obtain image segments in all directions centered on the mixing drum based on the images captured by multiple cameras, ensuring the integrity of the image segments in all directions centered on the mixing drum.

[0062] In one possible example, step A1 includes: acquiring a set of image frames captured by the multiple cameras at the current time node; repeatedly acquiring the set of image frames captured by the multiple cameras at preset time intervals to obtain multiple sets of continuous image frames; stitching together the image frames in the same direction centered on the stirring drum in the multiple sets of continuous image frames to obtain continuous image frames with a time axis in the same direction centered on the stirring drum; and determining the continuous image frames with a time axis in all directions centered on the stirring drum.

[0063] In practical applications, the preset time can be set according to the rotation speed of the stirring drum. The faster the rotation speed, the shorter the preset time, and the slower the rotation speed, the longer the preset time.

[0064] The collection of images captured by the multiple cameras at the current time point displays the road condition information around the vehicle in a 360-degree range at the current time point.

[0065] For example, multiple cameras include camera 1, camera 2, and camera 3. Camera 1 captures image 1, camera 2 captures image 2, and camera 3 captures image 3. If all directions centered on the mixing drum include direction 1, direction 2, direction 3, and direction 4, direction 1 refers to the right side of the mixing truck body, direction 2 refers to the left side of the mixing truck body, direction 3 refers to the top of the mixing truck body, and direction 4 refers to all directions centered on the mixing drum except for direction 1, direction 2, and direction 3. The vehicle domain controller acquires a set of image frames captured by multiple cameras at the current time node. At preset time intervals, it repeatedly acquires the set of image frames captured by the multiple cameras to obtain multiple sets of continuous image frames. The image frames in direction 1 from these multiple sets of continuous image frames are stitched together to obtain continuous image frames with a timeline in direction 1. The image frames in direction 2 from these multiple sets of continuous image frames are stitched together to obtain continuous image frames with a timeline in direction 2. The image frames in direction 3 from these multiple sets of continuous image frames are stitched together to obtain continuous image frames with a timeline in direction 3. The image frames in direction 4 from these multiple sets of continuous image frames are stitched together to obtain continuous image frames with a timeline in direction 4. Finally, continuous image frames with timelines in directions 1, 2, 3, and 4 are determined.

[0066] As can be seen, in this example, the vehicle domain controller can obtain a set of multiple image frames captured by multiple cameras at different time points based on the image images captured by multiple cameras, and obtain a set of continuous image frames with time axes in the same direction based on the image frames in the same direction in the set of multiple continuous image frames, and finally determine the continuous image frames with time axes in all directions to ensure the integrity of image segments in all directions centered on the mixing drum.

[0067] In one possible example, the target direction includes the left and right sides of the mixer truck. Determining whether there is a safety hazard event in the image segments in the target direction among all image segments in all directions includes: determining whether the image segments on the left and right sides of the mixer truck include an umbrella; if so, detecting the actual movement state of a first umbrella in the image segments on the left and right sides of the mixer truck, the actual movement state including moving or stationary; determining whether there is a second umbrella in the first umbrella that is in a moving state; if there is no second umbrella in the first umbrella, determining that there is no safety hazard event in the image segments on the left and right sides of the mixer truck.

[0068] It should be noted that since the mixer truck is in motion, the relative motion state of the umbrella surface is first calculated based on the image segments of the left and right sides of the mixer truck body. Then, the vehicle body domain controller can calculate the actual motion state of the umbrella surface based on the relative motion state.

[0069] For example, in the scenario where the cement mixer truck is driving slowly and turning in the rain, if a pedestrian is crossing the road with an umbrella in the direction of the turn and does not notice the cement mixer truck, the camera on the mixing drum can capture the umbrella surface and accurately identify the pedestrian based on the umbrella surface. That is, the image segment in the target direction may include at least one umbrella surface.

[0070] As can be seen in this example, the vehicle body domain controller of the mixer truck can determine that there are no moving umbrellas on the left and right sides of the mixer truck body based on the images captured by multiple cameras. This determines that there are no safety hazards on the left and right sides of the mixer truck body, improves the reliability of driving safety warnings, and enhances driving safety.

[0071] In one possible example, after determining whether there is a second umbrella surface in the first umbrella surface that is actually moving, the method further includes: if there is a second umbrella surface in the first umbrella surface, then determining whether there is a third umbrella surface in the second umbrella surface whose direction of movement intersects with the direction of movement of the mixer truck; if there is the third umbrella surface in the second umbrella surface, then determining that there is a safety hazard event in the image segments on the left and right sides of the mixer truck.

[0072] Furthermore, after determining whether there is a third umbrella surface in the second umbrella surface whose direction of movement intersects with the direction of movement of the mixer truck, if the second umbrella surface does not contain the third umbrella surface, it is determined that there is no safety hazard event in the image segments on the left and right sides of the mixer truck body.

[0073] For example, in a scenario where the cement mixer truck is driving slowly and turning on a sunny day, if a pedestrian is crossing the road with an umbrella in the direction of the turn and does not notice the cement mixer truck, the camera on the mixing drum can capture the umbrella surface and accurately identify the pedestrian based on the umbrella surface. That is, the image segment in the target direction may include at least one umbrella surface.

[0074] As can be seen in this example, the vehicle body domain controller of the mixer truck can determine that there is a safety hazard event on the left and right sides of the mixer truck body when the umbrella surface is actually moving and its direction of movement intersects with the direction of movement of the mixer truck, based on the images captured by multiple cameras. This allows the controller to issue a safety warning, improve the reliability of driving safety warnings, and enhance driving safety.

[0075] In one possible example, the target direction includes the top of the mixer truck. Determining whether a safety hazard exists in the image segments in the target direction among all image segments in all directions includes: determining the target distance from the mixer truck body to the top of the height restriction lane based on the image segments in the target direction; if the target distance is less than a first preset distance and greater than or equal to a second preset distance, then a safety hazard is determined to exist, where the second preset distance is less than the first preset distance; if the target distance is greater than or equal to the first preset distance, then no safety hazard is determined to exist.

[0076] The first preset distance can be 15cm, 12cm, 10cm, etc., and the second preset distance can be 20cm, 15cm, 12cm, etc. There are no specific limitations on the first and second preset distances.

[0077] For example, the first preset distance is 15cm and the second preset distance is 10cm. That is, when the distance between the cement mixer truck and the top of the height restriction lane is less than 15cm but greater than or equal to 10cm, there is a certain driving risk, that is, there is a safety hazard. The driver is reminded by the warning information that there is a risk of collision with the top of the lane and should drive with caution. When the distance between the cement mixer truck and the top of the height restriction lane is less than 10cm, the driver is reminded by the warning information that the lane is impassable and should replan the route.

[0078] As can be seen in this example, the vehicle body domain controller can determine the distance between the vehicle body and the top of the height restriction lane based on images captured by multiple cameras when the mixer truck is driving in a height restriction lane. This allows the controller to determine whether there is a safety hazard above the vehicle body and then issue a corresponding safety warning, thereby improving the accuracy of driving safety warnings and enhancing driving safety.

[0079] In one possible example, the warning device includes a voice broadcasting device and / or a display device, and the output of the warning information through the warning device includes: broadcasting the warning information through the voice broadcasting device; and / or displaying the warning information through the display device.

[0080] For example, in a scenario where a pedestrian is crossing with an umbrella, if the safety hazard involves the right side of the vehicle's umbrella surface, a text warning message such as "Caution: A pedestrian may be crossing the right side of the vehicle with an umbrella" can be broadcast via a voice announcement device, and / or, the same text warning message can be displayed via a display device, and / or, an image clip of the right side of the cement mixer truck's umbrella surface can be displayed via a display device. For example, please refer to [link to relevant documentation]. Figure 2d , Figure 2d This is a schematic diagram of a warning page provided in an embodiment of this application, such as... Figure 2d As shown, the warning page includes a text message that reads "Beware of pedestrians with umbrellas crossing on the right side of the vehicle" and an image clip of the umbrella on the right side of the cement mixer truck.

[0081] For another example, in the scenario of driving in a height-restricted lane, if the safety hazard involves the height of the section of road through which the cement mixer truck passes, a text message such as "There is a risk of top collision in the current driving lane, please drive with caution" can be broadcast through a voice broadcast device, and / or the text message "There is a risk of top collision in the current driving lane, please drive with caution" can be displayed through a display device, and / or an image clip of the top of the cement mixer truck can be displayed through a display device.

[0082] As can be seen in this example, when the mixer truck's body domain controller determines that there is a safety hazard, it can broadcast the corresponding warning information through the voice broadcast device in the warning device, and / or display the corresponding warning information through the display device, so that the mixer truck driver can clearly perceive the road conditions in the blind spot around the mixer truck, and can give early warning of emergency events, improve the reliability and effectiveness of driving safety warnings, and improve driving safety.

[0083] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0084] Consistent with the embodiments shown above, such as Figure 3a As shown, Figure 3a This is a block diagram of the functional units of a data processing device for safety hazard events during driving, provided in an embodiment of this application. Figure 3a In this context, a data processing device 300 for safety hazard events during vehicle operation is applied to the vehicle body domain controller of a concrete mixer truck. The mixer truck also includes a mixing drum and a warning device. Multiple cameras are installed on the mixing drum, and the image acquisition range of these cameras covers the entire angle centered on the mixing drum. The vehicle body domain controller is connected to the mixing drum, the multiple cameras, and the warning device. The data processing device 300 for safety hazard events during vehicle operation includes:

[0085] The acquisition unit 301 is used to acquire video images of the mixer truck driving process through the multiple cameras;

[0086] The determining unit 302 is used to determine, based on the image frame, that when the operating state of the stirring drum is rotation, the image frame determines image segments in all directions centered on the stirring drum.

[0087] The judgment unit 303 is used to judge whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions, and the target direction includes the left and right sides of the mixer truck or the top of the mixer truck.

[0088] The early warning information generation unit 304 is used to generate early warning information based on the safety hazard event if it exists, and the early warning information is used to indicate the existence of the safety hazard event;

[0089] The early warning unit 305 is used to output the early warning information through the early warning device to remind the driver of the mixer truck to pay attention to the safety hazard.

[0090] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0091] When using integrated units, such as Figure 3b As shown, Figure 3b This is a functional unit block diagram of another data processing device for safety hazard events during driving, provided in an embodiment of this application. Figure 3bIn the vehicle, the data processing device 310 for safety hazard events during driving is applied to the vehicle body domain controller of the mixer truck. The mixer truck also includes a mixing drum and an early warning device. Multiple cameras are installed on the mixing drum, and the image acquisition range of the multiple cameras covers all angles centered on the mixing drum. The vehicle body domain controller is connected to the mixing drum, the multiple cameras, and the early warning device respectively. The data processing device 310 for safety hazard events during driving includes a processing module 312 and a communication module 311.

[0092] The processing module 312 is used to acquire video images of the mixer truck's driving process via the multiple cameras through the communication module 311; determine that the mixing drum is rotating based on the video images; determine image segments in all directions centered on the mixing drum based on the video images; determine whether there is a safety hazard event in the image segments in the target direction among the image segments in all directions, where the target direction includes the left and right sides of the mixer truck or the top of the mixer truck; if so, generate warning information based on the safety hazard event, the warning information indicating the existence of the safety hazard event; and output the warning information through the warning device to remind the mixer truck driver to pay attention to the safety hazard event. The communication module 311 is used to support the interaction between the data processing device 310 for safety hazard events during driving and other devices. Figure 3b As shown, the data processing device 310 for safety hazard events during driving may further include a storage module 313, which is used to store the program code and data of the data processing device 310 for safety hazard events during driving.

[0093] The processing module 312 can be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication module 311 can be a transceiver, RF circuitry, or a communication interface, etc. The storage module 313 can be a memory.

[0094] All relevant content in each scenario involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here. The data processing device 310 for safety hazard events during driving can perform the above-mentioned... Figure 2a The data processing method shown is for safety hazard events during driving.

[0095] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, vehicle domain controller, or data center to another website, computer, vehicle domain controller, or data center via wired or wireless means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a vehicle domain controller or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0096] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. For example... Figure 4 As shown, the electronic device 400 may include one or more of the following components: a processor 401 and a memory 402 coupled to the processor 401, wherein the memory 402 may store one or more programs, which may be configured to implement the methods described in the above embodiments when executed by one or more processors 401. The electronic device 400 may be the vehicle body domain controller in the aforementioned mixer truck.

[0097] Processor 401 may include one or more processing cores. Processor 401 connects to various parts within the electronic device 400 using various interfaces and lines, and performs various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets, or instruction sets stored in memory 402, and by calling data stored in memory 402. Optionally, processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, passenger interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 401 and may be implemented separately using a communication chip.

[0098] The memory 402 may include random access memory (RAM) or read-only memory (ROM). The memory 402 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 402 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the electronic device 400 during use.

[0099] It is understood that the electronic device 400 may include more or fewer structural elements than those shown in the above block diagram, such as a power module, physical buttons, WiFi (Wireless Fidelity) module, speaker, Bluetooth module, sensor, etc., without limitation.

[0100] This application also provides a computer storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements some or all of the steps of any of the methods described in the above method embodiments.

[0101] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described in the first aspect of this application.

[0102] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0106] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, a vehicle domain controller, or a network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a portable hard disk, a magnetic disk, an optical disk, volatile memory, or non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM), etc., which are various media capable of storing program code.

[0107] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions without departing from the spirit and scope of the present invention, and various modifications and alterations can be made, including combinations of the different functions and implementation steps described above, as well as software and hardware implementation methods, all of which are within the protection scope of the present invention.

Claims

1. A data processing method for safety hazard events during driving, characterized in that, The method includes: Multiple cameras capture video footage of the mixer truck's operation. When the stirring drum is determined to be rotating based on the image, image segments in all directions centered on the stirring drum are determined based on the image. The stirring drum's operating state includes rotation and stillness. The image frames corresponding to the image segments in the same direction centered on the stirring drum are obtained by cutting and splicing image frames extracted from image images captured by multiple cameras. Determine whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions, wherein the target direction includes the left and right sides of the mixer truck or the top of the mixer truck. If such a safety hazard exists, an early warning message is generated based on the safety hazard event. The early warning message is used to indicate the existence of the safety hazard event. The early warning message is output through the early warning device to remind the driver of the mixer truck to pay attention to the safety hazard event. The step of determining image segments in all directions centered on the stirring drum based on the image frame includes: determining continuous image frames with time axes in all directions centered on the stirring drum based on the image frame; and generating the image segments based on the continuous image frames with time axes. The step of determining continuous image frames with time axes in all directions centered on the stirring drum based on the image footage includes: acquiring a set of image frames captured by the multiple cameras at the current time node; repeatedly acquiring the set of image frames captured by the multiple cameras at preset time intervals to obtain multiple sets of continuous image frames; stitching together the image frames in the same direction centered on the stirring drum in the multiple sets of continuous image frames to obtain continuous image frames with time axes in the same direction centered on the stirring drum; and determining continuous image frames with time axes in all directions centered on the stirring drum. The target direction includes the left and right sides of the mixer truck. Determining whether there is a safety hazard in the image segments in the target direction among all image segments in all directions includes: determining whether the image segments on the left and right sides of the mixer truck include an umbrella; if so, detecting the actual movement state of a first umbrella in the image segments on the left and right sides of the mixer truck, the actual movement state including moving or stationary; determining whether there is a second umbrella in the first umbrella that is in a moving state; if there is no second umbrella in the first umbrella, then determining that there is no safety hazard in the image segments on the left and right sides of the mixer truck.

2. The method according to claim 1, characterized in that, After determining whether there is a second umbrella surface in the first umbrella surface that is actually in a moving state, the method further includes: If the first umbrella surface exists, then determine whether there is a third umbrella surface in the second umbrella surface whose direction of movement intersects with the direction of movement of the mixer truck; If the third umbrella surface exists in the second umbrella surface, then it is determined that there is a safety hazard event in the image segments on the left and right sides of the mixer truck body.

3. The method according to claim 1, characterized in that, The target direction includes the area above the mixer truck. Determining whether a safety hazard event exists in the image segments in the target direction among the image segments from all directions includes: The target distance from the top of the height restriction channel to the body of the mixer truck is determined based on the image clip above the body of the mixer truck. If the target distance is less than a first preset distance and greater than or equal to a second preset distance, then a safety hazard event is determined to exist, wherein the second preset distance is less than the first preset distance; If the target distance is greater than or equal to the first preset distance, then it is determined that there is no safety hazard event in the image segment above the mixer truck.

4. The method according to any one of claims 1 to 3, characterized in that, The warning device includes a voice broadcasting device and / or a display device, and the output of the warning information through the warning device includes: broadcasting the warning information through the voice broadcasting device; and / or displaying the warning information through the display device.

5. The method according to claim 4, characterized in that, The warning information includes text prompts and / or video clips of the safety hazard event.

6. The method according to any one of claims 1 to 3, characterized in that, The plurality of cameras are mounted on the outer casing of the stirring drum, which is perpendicular to the rotation center line of the stirring drum and is located on any cross-section of the stirring drum within the mounting range.

7. A data processing device for safety hazard events during vehicle operation, characterized in that, The device includes: The acquisition unit is used to capture video footage of the mixer truck's driving process using multiple cameras; The determining unit is used to determine the operating state of the stirring drum as rotation based on the image frame, and to determine image segments in all directions centered on the stirring drum based on the image frame. The operating state of the stirring drum includes rotation and stationary. The image frames corresponding to the image segments in the same direction centered on the stirring drum are obtained by cutting and splicing image frames extracted from image frames captured by multiple cameras. The judgment unit is used to determine whether there is a safety hazard event in the image segment in the target direction among the image segments in all directions, and the target direction includes the left and right sides of the mixer truck or the top of the mixer truck. The early warning information generation unit is used to generate early warning information based on the safety hazard event if it exists, and the early warning information is used to indicate the existence of the safety hazard event; The early warning unit is used to output the early warning information through the early warning device to remind the driver of the mixer truck to pay attention to the safety hazard event; in, The step of determining image segments in all directions centered on the mixing drum based on the image frame includes: determining continuous image frames with time axes in all directions centered on the mixing drum based on the image frame; and generating the image segments based on the continuous image frames with time axes. The step of determining, based on the image footage, consecutive frame images with time axes in all directions centered on the stirring drum includes: Obtain the set of image frames captured by the multiple cameras at the current time node; repeatedly obtain the set of image frames captured by the multiple cameras at preset time intervals to obtain multiple sets of continuous image frames; stitch together the image frames in the same direction centered on the stirring drum in the multiple sets of continuous image frames to obtain continuous image frames with time axes in the same direction centered on the stirring drum; determine the continuous image frames with time axes in all directions centered on the stirring drum. The target direction includes the left and right sides of the mixer truck. Determining whether there is a safety hazard in the image segments in the target direction among all image segments in all directions includes: determining whether the image segments on the left and right sides of the mixer truck include an umbrella; if so, detecting the actual movement state of a first umbrella in the image segments on the left and right sides of the mixer truck, the actual movement state including moving or stationary; determining whether there is a second umbrella in the first umbrella that is in a moving state; if there is no second umbrella in the first umbrella, then determining that there is no safety hazard in the image segments on the left and right sides of the mixer truck.

8. An electronic device, characterized in that, It includes a processor and a memory, one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps in the method as claimed in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-6.

10. A computer program product, characterized in that, The computation contained in the computer program product enables the implementation of the method described in any one of claims 1-6 when the program is executed by a processor.

Images