A risk early warning method and related device

Abstract

The application provides a risk early warning method and related equipment. The risk early warning method comprises: splitting a to-be-detected image to obtain a plurality of image blocks; determining a first probability value of each image block containing a target, and determining a second probability value of the to-be-detected image containing the target according to each first probability value; and in the case that the second probability value is greater than a preset threshold, outputting prompt information according to the to-be-detected image, the prompt information being used for indicating at least one of a risk caused by the target, a position of the target, and a region where the target is located. In the application, the accuracy of risk early warning of the target is improved.

Description

Technical Field

[0001] This application relates to the field of risk warning technology, and in particular to a risk warning method and related equipment. Background Technology

[0002] Vehicle-mounted road environment perception and station inspection vision technology can be used to identify the status of targets such as water accumulation and obstacles in roads, factory passages and equipment operation areas, providing a basis for vehicle detour, speed limit warning, inspection and handling and safety management.

[0003] In exemplary techniques, images with different road backgrounds are typically acquired first and pixel-level annotated to form a semantic segmentation dataset. Then, an encoder-decoder segmentation network is constructed to extract regions containing targets from the input images and combine them with an output decision to provide target recognition results.

[0004] However, this method requires obtaining high-resolution images and then scaling them down to a fixed size before inputting them into the model. This results in small-area targets, distant targets, and elongated targets being weakened or even obscured due to the scaling down, making it impossible to identify targets in the image and leading to low accuracy in risk warnings. Summary of the Invention

[0005] Based on the aforementioned technological status, this application provides a risk warning method and related equipment to address the problem of low accuracy in risk warnings for targets.

[0006] To achieve the above-mentioned technical objectives, this application proposes the following technical solution: Firstly, this application provides a risk warning method, including: The image to be detected is segmented into multiple image blocks; A first probability value is determined for each of the image patches to contain the target, and a second probability value is determined for the image to be detected to contain the target based on each of the first probability values; If the second probability value is greater than a preset threshold, a prompt message is output based on the image to be detected. The prompt message is used to indicate at least one of the risks caused by the target, the location of the target, and the area where the target is located.

[0007] In some implementations, there is an overlapping region between adjacent image patches, and determining a first probability value that each image patch contains the target includes: The image patch is input into the target detection model to obtain the third probability value of each pixel in the image patch as the target; The fusion weight corresponding to the pixel is determined based on the distance between the target position of the pixel in the image block and the center position of the image block, wherein the distance and the fusion weight are negatively correlated. Based on the third probability value corresponding to each pixel in the image block and the fusion weight, a first probability value is determined for the image block to contain the target.

[0008] In some implementations, determining a second probability value that the image to be detected contains a target based on each of the first probability values ​​includes: The sum of each of the first probability values ​​is determined, and the sum of the fusion weights corresponding to each pixel in the image to be detected is determined; The ratio between the sum of the first probability values ​​and the sum of the fusion weights corresponding to each pixel in the image to be detected is determined and used as the second probability value.

[0009] In some implementations, the prompt information is used to indicate the risk posed by the target, and the step of outputting the prompt information based on the image to be detected includes: The image to be detected is converted to obtain a binary mask image; The intersection-union ratio is determined based on the region containing the target in the image to be detected and the region containing the target in the previous frame of the image to be detected. Based on a first area ratio between the area of ​​the binary mask image and the area of ​​the image to be detected; Perform connected component analysis on the binary mask image to obtain the first area of ​​the largest first connected region; An average area ratio is determined based on the first area ratio and a plurality of second area ratios, and an average area is determined based on the first area and a plurality of second areas. Each of the second area ratios is used to indicate the area ratio of the region containing the target in each of the other images in the video where the image to be detected is located. The second area is used to indicate the area of ​​the largest connected region of the other images. Each of the other images is sequentially adjacent to the image to be detected in the video. Based on the intersection-union ratio, the average area ratio, and the average area, a risk score is determined, and a target risk level is determined based on the risk score. A prompt message containing the target risk level is then output.

[0010] In some implementations, determining the risk score based on the intersection-union ratio, the average area percentage, and the average area includes: If a maximum connected region exists in the previous frame image, the centroid displacement parameter is determined based on the area of ​​the maximum connected region corresponding to the image to be detected and the area of ​​the maximum connected region corresponding to the previous frame image. The risk score is determined based on the centroid displacement parameter, the intersection-to-union ratio, the average area percentage, and the average area.

[0011] In some implementations, determining the first area percentage of the region containing the target in the image to be detected based on the binary mask image includes: Compare the intersection-union ratio with a set threshold; If the intersection-union ratio is greater than or equal to the set threshold, a first area percentage of the region containing the target in the image to be detected is determined based on the binary mask image.

[0012] In some implementations, after comparing the intersection-union ratio with a set threshold, the method further includes: If the cross-union ratio is greater than or equal to the set threshold, the target identified in the image to be detected is determined to be a false alarm target.

[0013] In some implementations, determining the target risk level based on the risk score includes: The level associated with the range of the risk score is taken as the initial risk level; The target risk level is determined based on the initial risk level and the historical risk level output from the previous frame image.

[0014] In some implementations, determining the target risk level based on the initial risk level and the historical risk level output from the previous frame image includes: When the initial risk level is medium risk level and the historical risk level is high risk level, a first difference between a first risk threshold and a first preset value is determined, wherein when the risk score is less than the first risk threshold and greater than or equal to a second risk threshold, the initial risk level of the image to be detected is medium risk level. If the risk score is greater than or equal to the first difference, the high-risk level is determined as the target risk level; If the risk score is less than the first difference, the medium risk level will be determined as the target risk level.

[0015] In some implementations, determining the target risk level based on the initial risk level and the historical risk level output from the previous frame image includes: When the initial risk level is low and the historical risk level is medium, a second difference between a second risk threshold and a second preset value is determined, wherein when the risk score is less than the second risk threshold, the initial risk level of the image to be detected is low. If the risk score is greater than or equal to the second difference, the medium risk level will be determined as the target risk level. If the risk score is less than the second difference, the low risk level is determined as the target risk level.

[0016] Secondly, this application provides a risk warning device, including a memory and a processor, wherein, The memory is connected to the processor and is used to store programs; The processor is used to implement the risk warning method as described in the first aspect or any implementation thereof by running the program in the memory.

[0017] Thirdly, this application provides a computer program product, including computer instructions, which, when executed by a processor, implement the risk warning method as described in the first aspect or any implementation thereof.

[0018] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the risk warning method as described in the first aspect or any implementation thereof.

[0019] This application provides a risk warning method and related device. The method involves segmenting an image to be detected into multiple image blocks, determining a first probability value for each image block containing a target, and determining a second probability value for the image containing a target based on each first probability value. If the second probability value is greater than a preset threshold, the method outputs at least one of the following warning information: the risk caused by the target, the target's location, and the area where the target is located. In this application, the image to be detected is segmented, and a second probability of the image containing a target is determined based on the second probability of each segmented image block containing a target. This allows for accurate determination of whether the image contains a target without reducing the image size, preventing small targets, distant targets, and elongated targets from being weakened or even obscured due to image reduction. This improves the accuracy of target detection and thus enhances the accuracy of risk warnings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1A flowchart of a risk warning method provided in this application embodiment Figure 1 .

[0022] Figure 2 A flowchart of a risk warning method provided in this application embodiment Figure 2 .

[0023] Figure 3 A flowchart of a risk warning method provided in this application embodiment Figure 3 .

[0024] Figure 4 A flowchart of a risk warning method provided in this application embodiment Figure 4 .

[0025] Figure 5 This is a schematic diagram of the functional modules of a risk warning device provided in an embodiment of this application.

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It should be noted that the user information (including but not limited to electrical equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0029] Vehicle-mounted road environment perception and station inspection vision technology can be used to identify the status of targets such as water accumulation and obstacles in roads, factory passages and equipment operation areas, providing a basis for vehicle detour, speed limit warning, inspection and handling and safety management.

[0030] In exemplary techniques, images with different road backgrounds are typically acquired first and pixel-level annotated to form a semantic segmentation dataset. Then, an encoder-decoder segmentation network is constructed to extract regions containing targets from the input images and combine them with an output decision to provide target recognition results.

[0031] First, existing deployment methods typically scale high-resolution images to a fixed size before inputting them into the model to meet computational constraints and inference efficiency requirements. However, for distant water accumulations, small-area water accumulations, edge water accumulations, and thin strips of water accumulation, scaling the entire image compresses local foreground details and weakens texture, edge, and regional morphological information, causing the model to miss or deviate from the boundaries of risk areas that should be identified during the deployment phase. Secondly, while some existing solutions divide high-resolution images into several sub-blocks for inference, they often lack an effective mechanism for fusion of overlapping regions. If the prediction results of different windows are directly stitched together at the edge regions, problems such as discontinuous probabilities at window boundaries, broken region contours, and repeated or omitted local water accumulation areas can easily occur. This is especially true when the water accumulation area is located at the window boundary, the area is small, or the edge shape is complex. In such cases, the stitching error will further affect the stability of the overall recognition result. Furthermore, existing vehicle-side or edge-side deployments typically determine the presence of water accumulation and risk level directly based on single-frame segmentation results, rarely utilizing the temporal continuity between consecutive frames. Under conditions such as nighttime, shadows, backlighting, slippery reflective surfaces, localized water stains, and vehicle vibration, single-frame output is prone to fluctuations due to instantaneous noise, resulting in frequent alarm level increases and decreases, high false alarm rates, and unstable on-site handling criteria. Finally, existing solutions typically handle high-resolution inference, whole-image reconstruction, and risk assessment in a decentralized manner, lacking a technical chain that is collaboratively designed around the stability of output during real-world deployment. In other words, although the model can provide pixel-level segmentation results after deployment, there is a lack of a unified technical control mechanism between how the input image is segmented, how overlapping regions are merged, how single-frame results are temporally smoothed, and how the final risk level is stably output. This makes it difficult for vehicle-side or edge systems to balance recognition accuracy, output stability, and engineering reproducibility in complex scenarios.

[0032] Therefore, this application provides a risk warning method, and the following detailed description of the risk warning method proposed in this application is provided through various embodiments.

[0033] Reference Figure 1 , Figure 1 A flowchart of a risk warning method provided in this application embodiment Figure 1 .like Figure 1 As shown, the risk warning method provided in this embodiment includes: Step S101: The image to be detected is segmented to obtain multiple image blocks.

[0034] In this embodiment, the executing entity is a risk warning device. For ease of description, the term "device" will be used to refer to the risk warning device below. The device can be any terminal device with target detection function, or it can be a component in a vehicle used for target detection.

[0035] Vehicle front-view cameras, surround-view cameras, inspection cameras, or fixed monitoring terminals are used at all times. The device acquires the RGB image of the current frame. This image is first stored in the onboard buffer module, edge buffer module, or server receive buffer. After acquiring the current frame image, the device preprocesses it to obtain the image to be detected. , 3 represents the size of the image to be detected, and 3 represents the number of image channels. Preprocessing includes decoding, normalization, color space conversion, and size detection. When the size of the image to be detected is larger than the preset size, the image will not be reduced in size; instead, it will be segmented into multiple image blocks. For example, a preset window size can be used. and sliding window step The image is segmented to obtain multiple image blocks. Let the first block be... The image blocks are denoted as Then the image patch can be represented as: ;in, Indicates the first The coordinates of the top left corner of the window; Preset window size; Characterize the image to be detected, i.e. .

[0036] Step S102: Determine a first probability value for each image patch containing the target, and determine a second probability value for the image to be detected containing the target based on each first probability value.

[0037] After obtaining each image patch, the probability value of each image patch containing a target is determined. The target could be, for example, puddles or an obstacle. A target detection model is set up in the device; this model can be a convolutional network, a Transformer network, or a combination of both, forming an encoder-decoder semantic segmentation model. The device inputs each image patch into the target detection model, and the model outputs the probability value of each image patch containing a target; this probability value is defined as a first probability value.

[0038] After obtaining each first probability value, a second probability value is determined based on each first probability value to determine if the image to be detected contains the target. For example, the highest first probability value is used as the second probability value for the image patch to contain the target.

[0039] Step S103: If the second probability value is greater than a preset threshold, output prompt information based on the image to be detected. The prompt information is used to indicate at least one of the risks caused by the target, the location of the target, and the area where the target is located.

[0040] After obtaining the second probability value, it is compared with a preset threshold. If the second probability value is greater than the preset threshold, it can be determined that the image to be detected contains a target. The device then outputs a prompt message based on the image to be detected. The prompt message includes at least one of the following: the risk posed by the target, the location of the target, and the region where the target is located. The risk posed by the target can be a risk level, which can be determined based on the second probability value, that is, the level associated with the interval of the second probability value is used as the risk level; the location and region of the target can be determined by the coordinates of the target in the image to be detected.

[0041] In this embodiment, the image to be detected is segmented into multiple image blocks, and a first probability value is determined for each image block containing a target. A second probability value is then determined based on each first probability value. If the second probability value is greater than a preset threshold, a warning message is output based on at least one of the following: the risk caused by the target, the target's location, and the area where the target is located. In this embodiment, the image to be detected is segmented, and a second probability of the target being contained in the image is determined based on the second probability of each segmented image block containing a target. This allows for accurate determination of whether the image contains a target without scaling down the image, preventing small targets, distant targets, and elongated targets from being weakened or even obscured due to image scaling, thus improving the accuracy of target detection and consequently, the accuracy of target risk warnings.

[0042] Reference Figure 2 , Figure 2 A flowchart of a risk warning method provided in this application embodiment Figure 2 ,based on Figure 1 In the embodiment shown, step S102 includes: Step S201: Input the image patch into the target detection model to obtain the third probability value of each pixel in the image patch being a target.

[0043] In this embodiment, the device inputs an image patch into the target detection model, and the target detection model outputs a probability value for each pixel in the image patch that is a target. This probability value is defined as a third probability value. For example, the target detection model is denoted as... Then the first The third probability value of a pixel in an image patch is represented as: ;in, This represents the predicted probability that pixel u within an image patch belongs to the target foreground. Pixel u can be one or more pixels within the image patch. Represents pixel u in an image block; This refers to the probability that a pixel u in an image patch is a target foreground element, as predicted by the model.

[0044] Step S202: Determine the fusion weight corresponding to the pixel based on the distance between the target position of the pixel in the image block and the center position of the image block, wherein the distance and the fusion weight are negatively correlated.

[0045] In this embodiment, there is an overlapping area between adjacent image blocks, that is, based on a preset window size. and sliding window step Segment the image with a step size that satisfies , This allows overlapping areas to be formed between image patches obtained from adjacent windows. This method can preserve detailed information about distant water accumulations, small-area water accumulations, and edge water accumulations with relatively high effective local resolution.

[0046] Since there are overlapping areas between image blocks, in order to prevent probability jumps caused by direct stitching of different image blocks, a fusion weight is configured for each pixel. The closer a pixel is to the center of the image block, the greater its fusion weight; the closer a pixel is to the boundary of the image block, the smaller its fusion weight. That is, the fusion weight adopts a distribution method of center enhancement and edge attenuation, so that the pixels in the central region of the image block contribute more to the fusion of the image block, and the direct stitching effect of the edge region of the image block is weakened.

[0047] In response, the device determines the fusion weight of a pixel based on the distance between the target position of the pixel in the image block and the center position of the image block, and the distance and the fusion weight are negatively correlated.

[0048] For example, the fusion weights can be expressed as: ω k (u)=max(0, (1-| - |) / (S w / 2))×max(0,(1-| - |) / (S h / 2)); where, For pixels coordinates For the first The center coordinates of each image patch.

[0049] Step S203: Determine the first probability value of the image block containing the target based on the third probability value corresponding to each pixel in the image block and the fusion weight.

[0050] After obtaining the fusion weights of the pixels, a first probability value can be determined based on the third probability value of each pixel in the image block and the fusion weight. For example, the third probability value and the fusion weight of each pixel in the image block are weighted and calculated to obtain a first value; the sum of the fusion weights of each pixel in the image block is determined, and the ratio between the first value and the fusion weight can be used as the first probability value.

[0051] Furthermore, when determining the second probability value based on each first probability value, the device determines the sum of each first probability value, that is, the sum of each first probability value is... The device then determines the sum of the fusion weights of each pixel in the image to be detected; the sum of the fusion weights is... The device determines the ratio between the sum of the first probability values ​​and the sum of the fusion weights, using this ratio as the second probability value. Obtaining the probability values ​​for the entire image in this way reduces segmentation artifacts at window boundaries and contour breaks.

[0052] In this embodiment, the image is divided into multiple image blocks with overlapping areas, and the target probability prediction is completed at the local window scale. In this way, local detail information of distant targets, small-area targets and edge targets can be preserved as much as possible without significantly increasing the input size of a single inference.

[0053] Figure 3 A flowchart of a risk warning method provided in this application embodiment Figure 3 ,based on Figure 1 or Figure 2 In the embodiment shown, step S103 includes: Step S301: Convert the image to be detected to obtain a binary mask image, and determine the intersection-union ratio based on the region containing the target in the image to be detected and the region containing the target in the previous frame of the image to be detected.

[0054] In this embodiment, the prompt information includes the risk caused by the target. When it is necessary to calculate the risk, the image to be detected needs to be converted into a binary mask image. A binary mask image can be represented as: , , This is the second probability value; that is, after obtaining the second probability value... Greater than or equal to the preset threshold The binary mask corresponding to the image to be detected =1 indicates that pixel u in the image to be detected is identified as the target foreground; Less than the preset threshold The binary mask corresponding to the image to be detected =0 indicates that any pixel u in the image to be detected is not the target foreground.

[0055] The image to be detected is the image of the target to be detected in the video, and therefore has a previous frame in the video. The device needs to calculate the intersection-union ratio (IUR) between the regions containing the target in the image to be detected and the regions containing the target in the previous frame to measure the temporal stability of the target regions. The IUR is expressed as... , The region in the image to be detected that contains the target. The crossover ratio is the ratio of the number of identical pixels with a foreground value of 1 in the two regions to the total number of pixels in the pixel union of the two regions.

[0056] Step S302: Based on the binary mask image, determine the first area ratio between the area of ​​the region containing the target in the image to be detected and the area of ​​the image to be detected, and perform connected component analysis on the binary mask image to obtain the first area of ​​the largest first connected region.

[0057] The device, based on a binary mask image, determines the ratio of the area of ​​the region containing the target in the image to the area of ​​the image to be detected. This ratio is defined as a first area ratio, which refers to the ratio between the number of pixels in the region containing the target and the number of pixels in the image to be detected. If the size of the image to be detected is... Then the proportion of the first area is expressed as: .

[0058] Furthermore, after obtaining the Cross-Union Ratio (CUI), the device can determine whether the target identification in the image to be detected is a continuously stable target region or a short-term abnormal output caused by instantaneous reflection, shadow disturbance, or local exposure change. The device compares the CUI with a set threshold. If the CUI is greater than or equal to the set threshold, the target identification is a continuously stable target region, and the risk caused by the target needs to be output. Therefore, the device performs a first area ratio determination of the region containing the target in the image to determine the risk caused by the candidate target. If the CUI is greater than or equal to the set threshold, the target identification in the image to be detected is determined to be a short-term abnormal output caused by instantaneous reflection, shadow disturbance, or local exposure change. In this case, the target identified in the image to be detected is determined to be a false alarm target, and the device will prompt an abnormality message indicating that the target identified in the image to be detected is abnormal.

[0059] After obtaining the first area ratio, the device performs connected component analysis on the binary mask image to obtain the area of ​​the largest first connected region, which is defined as the first area.

[0060] For example, performing connected component analysis on a binary mask image yields multiple connected regions. The first area of ​​the largest connected region. Defined as: .

[0061] Additionally, the centroid of the largest connected region can be determined, and this centroid is represented as: .

[0062] The aforementioned attributes can be used to characterize the range (first area percentage), size of major risk regions (largest connected region area), and spatial location (centroid) of the target region in the image to be detected.

[0063] Step S303: Determine the average area ratio based on the first area ratio and multiple second area ratios, and determine the average area based on the first area and multiple second areas. Each second area ratio is used to indicate the area ratio of the region containing the target in each other image in the video where the image to be detected is located. The second area is used to indicate the area of ​​the largest connected region in the other images. Each other image and the image to be detected are sequentially adjacent in the video.

[0064] In this embodiment, a spatiotemporal consistency score is introduced, that is, the temporal mean of the area is determined within a consecutive time window of N frames (i.e., multiple images). Normalized time-series mean of the largest connected component and continuity indicators For example, the device determines the average area percentage based on a first area percentage and a plurality of second area percentages. The average area percentage is obtained by averaging the sum of the first area percentage and multiple second area percentages. The average area is determined based on the first area and multiple second areas. The average area is obtained by averaging the sum of the first area and the sum of all the second areas. Each second area percentage indicates the area percentage of the target contained in each of the other images in the video containing the image to be detected. These other images are sequentially adjacent to the image to be detected in the video, meaning they form consecutive image frames. The second area also indicates the area of ​​the largest connected region among the other images.

[0065] Specifically, average area percentage Characterized as: average area Characterized as .

[0066] Step S304: Determine the risk score based on the intersection-to-merger ratio, average area ratio, and average area, determine the target risk level based on the risk score, and output a prompt message containing the target risk level.

[0067] Continuity indicators Determined by intersection-union ratio Represented as: , It refers to the cross-union ratio between the binary mask images of two adjacent frames in a video.

[0068] The device determines the risk score based on the crossover ratio, average area proportion, and average area.

[0069] In one example, risk score , These are the weighting coefficients.

[0070] In another example, if a maximum connected region exists in the previous frame, the centroid displacement parameter is determined based on the area of ​​the maximum connected region corresponding to the image to be detected and the area of ​​the maximum connected region corresponding to the previous frame. The centroid displacement parameter is characterized as follows: ,in, Let be the centroid of the largest connected region in the image to be detected. It is the centroid corresponding to the largest connected region in the previous frame image.

[0071] The device determines the risk score, i.e., the risk score, by using the centroid displacement parameter, crossover ratio, average area proportion, and average area. , These are the weighting coefficients.

[0072] After determining the risk score, the device determines the target risk level based on the risk score, and then outputs a prompt message containing the target risk level. For example, suppose the risk threshold meets... The system then according to Outputting three levels of risk: low, medium, and high: when Output low risk when; Outputting medium risk at the time; when High risk is generated at times.

[0073] In addition, the device can also output the binary mask of the water accumulation and the percentage of the water accumulation area in the current frame. Maximum connected region area Regional centroid and risk level This information can be directly sent to the driver's display terminal, inspection terminal, alarm module, or uploaded to the cloud management platform for early warning prompts, inspection verification, event recording, and subsequent dispatch.

[0074] In this embodiment, the area ratio of the target, the area of ​​the largest connected region, the centroid of the region, and the degree of mask overlap (intersection over union ratio) between consecutive frames are extracted to construct a time-stable risk score and graded output. Compared with the method of directly alarming based on the area threshold of a single frame, it can significantly reduce the risk level jump caused by instantaneous reflection, shadow disturbance and local exposure changes.

[0075] Figure 4 A flowchart of a risk warning method provided in this application embodiment Figure 4 .based on Figure 3 In the embodiment shown, step S304 includes: Step S401: The level associated with the interval where the risk score is located is taken as the initial risk level.

[0076] Step S402: Determine the target risk level based on the initial risk level and the historical risk level output from the previous frame image.

[0077] In this embodiment, a hysteresis determination mechanism is introduced to reduce risk level jumps. The device uses the level associated with the risk score interval as the initial risk level, and then determines the target risk level using the initial wind turbine level and the historical risk level output from the previous frame image.

[0078] In one example, the risk score is less than the first risk threshold. And greater than or equal to the second risk threshold In the case that, The initial risk level is medium risk. If the historical risk level is high risk, the device determines a first risk threshold. Compared with the first preset value The first difference between them, that is, the first difference is If the risk score is greater than or equal to the first difference... In cases where the risk score is less than the first difference, the high-risk level is determined as the target risk level; In this case, the medium-risk level will be set as the target risk level; that is, if a high-risk level was already identified in the previous timeframe, and the current level is... Slightly lower But higher If the current risk level is high, then output a high risk level; if the current risk level is low, then output a high risk level. Below If so, it is considered medium risk; furthermore, if the current... continuous Frames below If so, it is classified as a medium-risk level.

[0079] In another example, the risk score is less than the second risk threshold. In the case that, The initial risk level is low. If the historical risk level is medium, the device determines a second risk threshold. With the second preset value The second difference between them, that is, the second difference is If the risk score is greater than or equal to the second difference In cases where the risk score is less than the first difference, the medium risk level is determined as the target risk level; In this case, the low-risk level is set as the target risk level; that is, if the previous time step already outputs a medium-risk level, and the current time step is... Slightly lower But higher If the current risk is medium, then output medium risk; if the current risk is medium. Below If so, it is considered a low-risk level; furthermore, if the current continuous Frames below If so, it is considered a low-risk level.

[0080] It should be noted that the above-mentioned high-risk, medium-risk, and low-risk levels are three risk levels used to measure the degree of risk. The high-risk level is equivalent to the first level, the medium-risk level is equivalent to the second level, and the low-risk level is equivalent to the third level. That is, the risk level of the first level is higher than that of the second level, and the risk level of the second level is higher than that of the third level.

[0081] In another example, for an initial risk level of high risk, regardless of the historical risk level, the high risk level is determined as the target risk level.

[0082] In this embodiment, a threshold linkage and hysteresis judgment mechanism is proposed. When outputting the risk level, not only the risk score of the current frame is considered, but also the risk status of the previous moment or several consecutive frames are combined to make the risk level more stable in short-term fluctuation scenarios. In this way, false alarms and frequent switching caused by single-frame jitter can be reduced.

[0083] Based on the above embodiments, taking water accumulation as an example, this application has the following advantages: 1) This paper proposes a sliding window inference mechanism for small target detail preservation in high-resolution images. Unlike existing methods that uniformly scale the entire image before performing a single inference, this application divides the high-resolution input image into multiple image blocks with overlapping regions and performs water accumulation probability prediction at the local window scale. In this way, local detail information of distant water accumulation, small-area water accumulation, and edge water accumulation can be preserved as much as possible without significantly increasing the input size of a single inference.

[0084] 2) A probabilistic fusion and whole-image reconstruction mechanism for overlapping regions is proposed. Unlike existing schemes that directly generate the whole-image result by hard stitching after block inference, this application performs weighted fusion of the probability predictions of different windows in the overlapping region. By increasing the contribution of the prediction results in the window center region and reducing the direct stitching influence of the prediction results in the window edge region, it can effectively reduce segmentation breaks, contour misalignment and local jumps at the window boundary, and improve the continuity of the whole-image probability map and the stability of the segmentation boundary.

[0085] 3) This application proposes a risk output mechanism based on area proportion, maximum connected region, and temporal continuity. It not only outputs a single-frame water accumulation segmentation mask but also extracts the water accumulation area proportion, the area of ​​the maximum connected region, the region centroid, and the mask overlap between consecutive frames to construct a temporally stable risk score and graded output. Compared to methods that directly trigger alarms based solely on single-frame area thresholds, this invention significantly reduces risk level jumps caused by instantaneous reflections, shadow disturbances, and local exposure changes.

[0086] 4) This application proposes a threshold linkage and hysteresis judgment mechanism for vehicle-side deployment. When outputting risk levels, this application considers not only the risk score of the current frame but also the risk status of the previous moment or several consecutive frames, setting upgrade thresholds, maintenance thresholds, and fallback thresholds for each risk level, thus improving the stability of risk levels under short-term fluctuation scenarios. In this way, the system can reduce false alarms and frequent switching caused by single-frame jitter.

[0087] Therefore, this application does not simply deploy an existing segmentation model to the vehicle and add a single-frame area threshold alarm. Instead, it constructs a water accumulation recognition deployment technology chain centered on high-resolution sliding window inference, overlapping region probabilistic fusion, and spatiotemporal consistent risk output. Essentially, it integrates three deployment challenges—high-resolution small target preservation, continuous full-image output, and stable risk judgment—into a quantifiable, implementable, and deployable technical solution. This ensures deployment efficiency while improving its recognition stability, result continuity, and engineering practicality in real-world vehicle scenarios, factory inspection scenarios, and complex road environments.

[0088] Corresponding to the risk warning method described above, this application also provides a risk warning device. Figure 5 This is a schematic diagram of a risk warning device provided in an embodiment of this application. The risk warning device 500 provided in this embodiment includes: The segmentation module 510 is used to segment the image to be detected into multiple image blocks; The determination module 520 is used to determine a first probability value that each image patch contains a target, and to determine a second probability value that the image to be detected contains a target based on each first probability value; The output module 530 is used to output prompt information based on the image to be detected when the second probability value is greater than a preset threshold. The prompt information is used to indicate at least one of the risks caused by the target, the location of the target, and the area where the target is located.

[0089] In some implementations, the risk warning device 500 is also used for: The image patch is input into the target detection model to obtain the third probability value of each pixel in the image patch being a target; The fusion weight of a pixel is determined based on the distance between the target position of the pixel in the image block and the center position of the image block. The distance and the fusion weight are negatively correlated. Based on the third probability value corresponding to each pixel in the image patch and the fusion weight, the first probability value of the image patch containing the target is determined.

[0090] In some implementations, the risk warning device 500 is also used for: Determine the sum of the first probability values ​​and the sum of the fusion weights corresponding to each pixel in the image to be detected; The ratio between the sum of the first probability values ​​and the sum of the fusion weights corresponding to each pixel in the image to be detected is determined and used as the second probability value.

[0091] In some implementations, the risk warning device 500 is also used for: The image to be detected is transformed to obtain a binary mask image, and the intersection-union ratio is determined based on the regions containing the target in the image to be detected and the regions containing the target in the previous frame of the image to be detected. Based on the binary mask image, determine the first area ratio between the area of ​​the region containing the target in the image to be detected and the area of ​​the image to be detected, and perform connected component analysis on the binary mask image to obtain the first area of ​​the largest first connected region. The average area ratio is determined based on the first area ratio and multiple second area ratios. The average area is determined based on the first area and multiple second areas. Each second area ratio is used to indicate the area ratio of the region containing the target in each other image in the video where the image to be detected is located. The second area is used to indicate the area of ​​the largest connected region in other images. Each other image and the image to be detected are sequentially adjacent in the video. Based on the intersection-to-merger ratio, average area ratio, and average area, a risk score is determined, and a target risk level is determined based on the risk score. A prompt message containing the target risk level is then output.

[0092] In some implementations, the risk warning device 500 is also used for: If a maximum connected region exists in the previous frame image, the centroid displacement parameter is determined based on the area of ​​the maximum connected region corresponding to the image to be detected and the area of ​​the maximum connected region corresponding to the previous frame image. The risk score is determined based on the centroid displacement parameters, intersection-to-exchange ratio, average area percentage, and average area.

[0093] In some implementations, the risk warning device 500 is also used for: Compare crossover and union with a set threshold; If the cross-union ratio is greater than or equal to a set threshold, the first area proportion of the region containing the target in the image to be detected is determined based on the binary mask image.

[0094] In some implementations, the risk warning device 500 is also used for: If the cross-union ratio is greater than or equal to a set threshold, the target identified in the image to be detected is determined to be a false alarm target.

[0095] In some implementations, the risk warning device 500 is also used for: The initial risk level is the level associated with the range of risk scores. The target risk level is determined based on the initial risk level and the historical risk level output from the previous frame image.

[0096] In some implementations, the risk warning device 500 is also used for: When the initial risk level is medium risk level and the historical risk level is high risk level, a first difference between the first risk threshold and the first preset value is determined. Wherein, when the risk score is less than the first risk threshold and greater than or equal to the second risk threshold, the initial risk level of the image to be detected is medium risk level. If the risk score is greater than or equal to the first difference, the high-risk level will be determined as the target risk level. If the risk score is less than the first difference, the medium risk level will be determined as the target risk level.

[0097] In some implementations, the risk warning device 500 is also used for: When the initial risk level is low and the historical risk level is medium, a second difference between the second risk threshold and the second preset value is determined. When the risk score is less than the second risk threshold, the initial risk level of the image to be detected is low. If the risk score is greater than or equal to the second difference, the medium risk level will be determined as the target risk level. If the risk score is less than the second difference, the low-risk level will be determined as the target risk level.

[0098] The risk warning device and the risk warning method provided in the above embodiments of this application belong to the same application concept and can execute the risk warning method provided in any of the above embodiments of this application. They have the corresponding functional modules and beneficial effects for executing the risk warning method. Technical details not described in detail in this embodiment can be found in the specific processing content of the risk warning method provided in the above embodiments of this application, and will not be repeated here.

[0099] The functions implemented by each module in the risk warning device can be implemented by the same or different processors, and this application embodiment does not limit this.

[0100] It should be understood that the modules in the above risk warning device can be implemented by a processor calling firmware. For example, the system includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each module of the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal to the device or external to the system. Alternatively, the modules in the system can be implemented as hardware circuits. By designing the hardware circuits, some or all of the module functions can be implemented. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above modules are implemented by designing the logical relationships of the components within the circuit. In another implementation, the hardware circuit can be implemented by a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above modules. All modules of the above risk warning device can be implemented entirely by a processor calling firmware, entirely by hardware circuits, or partially by a processor calling firmware with the remaining parts implemented by hardware circuits.

[0101] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0102] As can be seen, each module in the above risk warning device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types.

[0103] Furthermore, the modules in the above-mentioned risk warning device can be integrated in whole or in part, or they can be implemented independently. In one implementation, these modules are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the modules of the device. The at least one processor can be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0104] This application provides a schematic diagram of the structure of an electronic device, see [link]. Figure 6 As shown, the electronic device includes a memory 600 and a processor 610; wherein the memory 600 is connected to the processor 610 and is used to store programs; the processor 610 is used to implement the risk warning method disclosed in any of the above embodiments by running the programs stored in the memory 600.

[0105] Specifically, the aforementioned electronic device may further include: a bus, a communication interface 620, an input device 630, and an output device 640. The electronic device may also include a data transceiver module, an image monitoring module, and a signal monitoring module.

[0106] The processor 610, memory 600, communication interface 620, input device 630, and output device 640 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components in an electronic device.

[0107] The processor 610 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0108] The processor 610 may include a main processor, as well as a baseband chip, modem, etc.

[0109] The memory 600 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 600 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0110] Input device 630 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0111] Output device 640 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0112] The communication interface 620 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0113] The processor 610 executes the program stored in the memory 600 and calls other devices, which can be used to implement any of the steps of the risk warning method provided in the above embodiments of this application.

[0114] It should be noted that the electronic device can be an in-vehicle terminal, a mobile phone, a wearable device, or a server, etc.; or it can be a vehicle that includes an in-vehicle terminal, etc.

[0115] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in the memory through the data interface to execute the risk warning method described in any of the above embodiments. For the specific processing procedure and its beneficial effects, please refer to the embodiments of the risk warning method described above.

[0116] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the risk warning methods according to various embodiments of this application as described in any of the above embodiments of this specification.

[0117] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the power device, as a standalone firmware package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor to perform the steps of the risk warning method according to various embodiments of this application described in any of the above embodiments of this specification, specifically implementing the steps of the risk warning method as described above.

[0119] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0120] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0121] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0122] The units of the apparatus in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0123] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0124] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0125] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or as firmware functional modules or sub-modules.

[0126] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer firmware, or a combination of both. To clearly illustrate the interchangeability of hardware and firmware, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or firmware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly using hardware, firmware units executed by a processor, or a combination of both. The firmware unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0128] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0129] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A risk warning method, characterized in that, include: The image to be detected is segmented into multiple image blocks; A first probability value is determined for each of the image patches to contain the target, and a second probability value is determined for the image to be detected to contain the target based on each of the first probability values; If the second probability value is greater than a preset threshold, a prompt message is output based on the image to be detected. The prompt message is used to indicate at least one of the risks caused by the target, the location of the target, and the area where the target is located.

2. The risk warning method according to claim 1, characterized in that, There are overlapping regions between adjacent image patches, and determining a first probability value that each image patch contains the target includes: The image patch is input into the target detection model to obtain the third probability value of each pixel in the image patch as the target; The fusion weight corresponding to the pixel is determined based on the distance between the target position of the pixel in the image block and the center position of the image block, wherein the distance and the fusion weight are negatively correlated. Based on the third probability value corresponding to each pixel in the image block and the fusion weight, a first probability value is determined for the image block to contain the target.

3. The risk warning method according to claim 2, characterized in that, Determining a second probability value for the image to be detected containing a target based on each of the first probability values ​​includes: The sum of each of the first probability values ​​is determined, and the sum of the fusion weights corresponding to each pixel in the image to be detected is determined; The ratio between the sum of the first probability values ​​and the sum of the fusion weights corresponding to each pixel in the image to be detected is determined and used as the second probability value.

4. The risk warning method according to claim 1, characterized in that, The prompt information is used to indicate the risk caused by the target, and the step of outputting the prompt information based on the image to be detected includes: The image to be detected is converted to obtain a binary mask image; The intersection-union ratio is determined based on the region containing the target in the image to be detected and the region containing the target in the previous frame of the image to be detected. Based on the binary mask image, a first area ratio is determined between the area of ​​the region containing the target in the image to be detected and the area of ​​the image to be detected. Perform connected component analysis on the binary mask image to obtain the first area of ​​the largest first connected region; An average area ratio is determined based on the first area ratio and a plurality of second area ratios, and an average area is determined based on the first area and a plurality of second areas. Each of the second area ratios is used to indicate the area ratio of the region containing the target in each of the other images in the video where the image to be detected is located. The second area is used to indicate the area of ​​the largest connected region of the other images. Each of the other images is sequentially adjacent to the image to be detected in the video. Based on the intersection-union ratio, the average area ratio, and the average area, a risk score is determined, and a target risk level is determined based on the risk score. A prompt message containing the target risk level is then output.

5. The risk warning method according to claim 4, characterized in that, The process of determining the risk score based on the intersection-union ratio, the average area percentage, and the average area includes: If a maximum connected region exists in the previous frame image, the centroid displacement parameter is determined based on the area of ​​the maximum connected region corresponding to the image to be detected and the area of ​​the maximum connected region corresponding to the previous frame image. The risk score is determined based on the centroid displacement parameter, the intersection-to-union ratio, the average area percentage, and the average area.

6. The risk warning method according to claim 4, characterized in that, Determining the first area percentage of the region containing the target in the image to be detected based on the binary mask image includes: Compare the intersection-union ratio with a set threshold; If the intersection-union ratio is greater than or equal to the set threshold, a first area percentage of the region containing the target in the image to be detected is determined based on the binary mask image.

7. The risk warning method according to claim 6, characterized in that, After comparing the intersection-union ratio with the set threshold, the method further includes: If the cross-union ratio is greater than or equal to the set threshold, the target identified in the image to be detected is determined to be a false alarm target.

8. The risk warning method according to claim 4, characterized in that, The process of determining the target risk level based on the risk score includes: The level associated with the range of the risk score is taken as the initial risk level; The target risk level is determined based on the initial risk level and the historical risk level output from the previous frame image.

9. The risk warning method according to claim 8, characterized in that, Determining the target risk level based on the initial risk level and the historical risk level output from the previous frame image includes: When the initial risk level is medium risk level and the historical risk level is high risk level, a first difference between a first risk threshold and a first preset value is determined, wherein when the risk score is less than the first risk threshold and greater than or equal to a second risk threshold, the initial risk level of the image to be detected is medium risk level. If the risk score is greater than or equal to the first difference, the high-risk level is determined as the target risk level; If the risk score is less than the first difference, the medium risk level will be determined as the target risk level.

10. The risk warning method according to claim 8, characterized in that, Determining the target risk level based on the initial risk level and the historical risk level output from the previous frame image includes: When the initial risk level is low and the historical risk level is medium, a second difference between a second risk threshold and a second preset value is determined, wherein when the risk score is less than the second risk threshold, the initial risk level of the image to be detected is low. If the risk score is greater than or equal to the second difference, the medium risk level will be determined as the target risk level. If the risk score is less than the second difference, the low risk level is determined as the target risk level.

11. A risk warning device, characterized in that, Including memory and processor, among which, The memory is connected to the processor and is used to store programs; The processor is used to implement the risk warning method as described in any one of claims 1-10 by running the program in the memory.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the risk warning method as described in any one of claims 1-10.

13. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the risk warning method as described in any one of claims 1-10.

Images