A monitoring device early warning method and system based on an improved image recognition model

Abstract

This application provides a monitoring equipment early warning method and system based on an improved image recognition model. The method includes the following steps: acquiring effective image information of a target traction device using monitoring equipment; obtaining an anomaly detection score of the target traction device using an improved image recognition model based on the effective image information, and using this score as monitoring status information; deploying the improved image recognition model on a cloud server, a local server, or through an edge cloud collaborative architecture; determining whether an anomaly has occurred in the load of the target traction device based on the anomaly detection score, and generating and sending early warning information to the user device when an anomaly occurs. This application does not rely on the computing power of the traction device itself, reducing the hardware requirements of the transportation device, thereby reducing the overall monitoring and early warning cost. The image recognition model can quickly process image information, promptly detect anomalies and issue early warnings, and does not rely on the status of the transportation device itself, enhancing the real-time performance and reliability of monitoring.

Description

[0001] This application claims partial priority to Chinese patent application No. CN202510388556.0, filed on March 31, 2025, the contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the technical fields of computer vision and image recognition, and in particular to a monitoring equipment early warning method and system based on an improved image recognition model. Background Technology

[0003] As automated logistics and smart factories increasingly demand automation and intelligence in cargo loading, unloading, and transportation processes, monitoring abnormal situations during transportation becomes essential.

[0004] To address the need for monitoring abnormal situations during cargo transportation, a trailer-mounted cargo drop detection system is provided. This system includes a human-machine interface (HMI) system mounted on the tractor unit, which communicates with sensors installed on multiple cargo-carrying trailer floorboards. During tractor unit operation, the HMI system receives monitoring results from the sensors, indicating whether cargo has fallen from the monitored location. This allows the HMI system to promptly alert the driver, enabling them to check and confirm whether cargo has fallen from the trailer. By monitoring cargo drop conditions through sensors and transmitting the results to the HMI system, the tractor unit driver can receive timely information during operation, preventing situations where cargo has fallen from the trailer without prompt detection.

[0005] However, the monitoring process relies on the computing power of the tractor itself, resulting in a large volume of communication and data processing. In other words, the relevant technology for monitoring anomalies in goods during transportation primarily depends on the transportation equipment itself, leading to high costs. Furthermore, it cannot effectively monitor malfunctions in the transportation equipment.

[0006] Therefore, there is an urgent need to provide a monitoring equipment early warning method and system based on an improved image recognition model to enhance the decoding success rate. Summary of the Invention

[0007] To address the aforementioned technical problems, this application provides a monitoring equipment early warning method and system based on an improved image recognition model.

[0008] The objective of this application is achieved through the following technical solution: Firstly, this application provides a monitoring equipment early warning method based on an improved image recognition model, the method comprising the following steps: S1, Use monitoring equipment to obtain effective image information of the target traction equipment; S2, based on the effective image information, use an improved image recognition model to obtain an anomaly detection score for the target traction device and use it as monitoring status information; the improved image recognition model is deployed on a cloud server, or on a local server, or through an edge cloud collaborative architecture; S3, based on the anomaly detection score, determine whether the load of the target traction device is abnormal, and generate an early warning message and send it to the user equipment when an anomaly occurs; The improved image recognition model includes an image segmentation model based on a deep neural network and a detection and scoring model driven by a rule engine; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain an anomaly detection score based on the target features.

[0009] The beneficial effects of this technical solution are that it does not rely on the computing power of the traction equipment itself, reducing the hardware requirements of the transportation device and thus lowering the overall monitoring and early warning costs. The image recognition model can quickly process image information, promptly detect anomalies and issue early warnings, and does not depend on the status of the transportation device itself. Compared with related technologies that rely on the transportation device itself for monitoring, resulting in large data processing volumes and low efficiency, this enhances the real-time performance and reliability of monitoring.

[0010] In some possible implementations, the target traction device includes a tractor and a trailer, and step S2 includes: Based on the effective image information, the main structural features of the tractor and the cargo area features of the trailer are extracted by the image segmentation model and used as target features in the effective image information; An anomaly detection strategy is obtained based on the characteristics of the body structure; based on the anomaly detection strategy and the characteristics of the loading area, an anomaly detection score is obtained using the detection scoring model and used as monitoring status information; Step S3 includes: when the anomaly detection score is not higher than the first threshold, it is considered that the load of the target traction device is abnormal, a first warning message is generated and sent to the user device.

[0011] The beneficial effects of this technical solution are that it dynamically selects anomaly judgment strategies based on the tractor's own status, avoiding misjudgments caused by a one-size-fits-all approach. It employs phased feature processing, independently extracting features from both the tractor and the cargo in the trailer, preventing feature extraction failures due to obstructions (such as cargo obscuring the license plate). Furthermore, it differentiates the severity of anomalies through multi-level thresholds, avoiding frequent false alarms that consume significant operational resources.

[0012] In some possible implementations, the monitoring equipment includes a plurality of first acquisition devices located along the operating route of the target traction equipment and a plurality of second acquisition devices located in a plurality of cargo loading areas along the route. Step S1 includes: using a first acquisition device to acquire first image information of the target traction device along its travel route, and using a second acquisition device to acquire second image information of the target traction device in the cargo loading area; Step S3 further includes: when the anomaly detection score is between a first threshold and a second threshold, obtaining preset loading information based on the second image information; when the second threshold is greater than the first threshold; obtaining loading area prediction information for the trailer based on the preset loading information and the current position information of the target traction device; determining whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information; when it is determined that the trailer is incorrectly loaded, generating a second warning message and sending it to the user device; or, Step S3 further includes: obtaining preset loading information based on the second image information; the second threshold being greater than the first threshold; simultaneously, continuously acquiring video of the loading area to obtain its temporal stability features; and fusing the temporal stability features with the spatial features obtained from the image segmentation model as the target features.

[0013] The beneficial effects of this technical solution are as follows: By setting up multiple data acquisition devices along the operating route and in the cargo loading area of ​​the traction equipment, comprehensive monitoring of the traction equipment at different locations and stages is achieved. This multi-point monitoring approach enables timely detection of anomalies during the traction equipment's movement and loading process, improving the comprehensiveness and accuracy of monitoring. Two thresholds are set, and different levels of early warning information are generated based on anomaly detection scores. This dual early warning mechanism can more flexibly respond to anomalies of varying degrees. For example, when an anomaly is severe, a first early warning is generated promptly to remind the user to take immediate action; when an anomaly is at a critical state, further analysis of loading information is conducted to determine if there are any loading errors, and a second early warning is generated, allowing the user to check the loading status in a timely manner. Through real-time monitoring and early warning of the traction equipment's cargo status, anomalies can be detected and handled promptly, reducing transportation accidents caused by cargo falling or improper loading, improving the safety of the transportation process, and reducing transportation risks and losses.

[0014] In some possible implementations, the second image information is QR code image information; the method of obtaining the second image information of the target traction device in the cargo loading area includes: using a second acquisition device to obtain the second image information from an electronic QR code screen set at the tail or sides of the trailer.

[0015] The advantages of this technical solution are: by using QR code image information for monitoring, it reduces the reliance on the hardware of the traction equipment itself, thus lowering the hardware cost of the monitoring system. Equipping the trailer with an electronic QR code screen is relatively inexpensive and facilitates integration and widespread adoption.

[0016] In some possible implementations, the step of obtaining an anomaly judgment strategy based on ontological structural features; and obtaining an anomaly detection score using the detection scoring model based on the anomaly judgment strategy and the features of the carrying area, and using this score as monitoring status information, includes: Based on the type and value of the ontology structure features, the rule engine matches anomaly detection strategies and weight coefficients from the pre-set strategy library. The features of the carrying area are input into the anomaly judgment strategy, and the anomaly detection score is calculated according to the deduction items and weight coefficients; the anomaly detection score is used as monitoring status information.

[0017] The beneficial effects of this technical solution are as follows: Through a rule engine, extracted features are evaluated and scored according to predefined rules and logic. The flexibility and interpretability of the rule engine allow anomaly detection strategies to be easily adjusted and optimized, enabling rapid adaptation to different application scenarios and needs.

[0018] In some possible implementations, obtaining the predicted loading area information of the trailer based on the preset loading information and the current location information of the target traction device includes: The loading plan for the target traction equipment is obtained from the preset loading information, the loading plan including the loading sequence and placement requirements of the goods in different loading areas; Based on the loading plan and current location information, the previous loading area reached by the target traction equipment is taken as the target loading area; Based on the cargo information of the target loading area in the loading plan, predict the types, quantities, and arrangement of cargo in the trailer's loading area, and generate loading area prediction information.

[0019] The beneficial effects of this technical solution are as follows: By acquiring the current location information of the traction equipment in real time, it can dynamically determine the loading area the traction equipment has reached. Loading area prediction information is only generated when the anomaly detection score is between the first and second thresholds, reducing data processing. This reduction in data processing allows for more efficient use of computing resources and lower energy consumption. Furthermore, it can predict the loading and unloading status of the trailer before entering the next loading area, helping users to identify potential loading problems in advance and reminding them to instruct loading personnel to check and handle issues at the next loading area, thus reducing transportation accidents caused by improper loading.

[0020] In some possible implementations, determining whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information includes: The loading similarity between the predicted loading area information and the first image information is obtained. When the loading similarity value is less than the preset similarity value, the loading is considered to be incorrect; otherwise, the loading is considered to be normal, and the counting begins and the counting count is incremented by one. When the counting count is greater than the preset count, the counting count is reset to zero, and a second threshold anomaly information is sent to the user equipment. The second threshold anomaly information is used to prompt the user to lower the value of the second threshold.

[0021] The beneficial effects of this technical solution are as follows: when the number of statistical attempts exceeds a preset limit, the user is prompted to lower the second threshold, which helps the user adjust parameters according to the actual situation, improving the adaptability and accuracy of the method. The alert is only triggered when the similarity is below the threshold, reducing unnecessary data processing and improving the execution efficiency of the method.

[0022] Secondly, this application also provides a monitoring equipment early warning device based on an improved image recognition model, comprising: The information acquisition module is used to acquire effective image information of the target traction equipment using monitoring equipment; The model processing module is used to obtain the monitoring status information of the target traction device based on the effective image information and using an improved image recognition model. The information sending module is used to generate early warning information and send it to the user equipment when the monitoring status information indicates that the load of the target traction device is abnormal; The improved image recognition model includes an image segmentation model and a detection and scoring model; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain monitoring status information based on the target features.

[0023] Thirdly, this application also provides an early warning system, including the monitoring equipment early warning device described in the second aspect, and a monitoring device for acquiring effective image information of the target traction device.

[0024] In some possible implementations, the monitoring equipment includes a plurality of first acquisition devices located along the operating route of the target traction equipment and a plurality of second acquisition devices located in a plurality of cargo loading areas along the route. Attached Figure Description

[0025] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0026] Figure 1 This is a flowchart illustrating a monitoring equipment early warning method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a process for obtaining monitoring status information provided in an embodiment of this application; Figure 3This is a partial flowchart illustrating the generation of early warning information provided in an embodiment of this application; Figure 4 This is a schematic diagram of a monitoring and early warning device provided in an embodiment of this application; Figure 5 This is a schematic diagram of a module of an early warning system provided in an embodiment of this application; Figure 6 This is a flowchart illustrating another monitoring device early warning method provided in an embodiment of this application. Detailed Implementation

[0027] First, a brief description of the technical field and related terms of the embodiments of this application will be given to facilitate understanding by those skilled in the art.

[0028] A surveillance camera array refers to a camera system composed of multiple cameras that simultaneously capture images from multiple perspectives to provide more comprehensive and detailed scene information. These cameras are, for example, mounted on a flat surface to ensure they can simultaneously capture image information at various points along the path of the target traction equipment.

[0029] A QR code is a black and white image with specific geometric shapes arranged in a certain pattern on a plane (two-dimensional direction). It can store a large amount of information and can be quickly read by scanning devices, thereby realizing the automatic collection and input of information.

[0030] In recent years, image recognition technology has been widely used in industrial monitoring and early warning systems, but existing technologies still have shortcomings when dealing with complex scenarios. One reason is that the monitoring process relies on the computing power of the tractor itself, resulting in a large volume of communication and data processing data. This makes effective monitoring impossible when the transport equipment malfunctions. Furthermore, to address these issues, related technologies primarily reduce the dimensions of information source analysis and processing, thereby decreasing data communication and processing volume. In other words, they rely on analyzing single images and lack the motivation to consider multi-source information.

[0031] To address the aforementioned challenges, this application proposes a monitoring equipment early warning method and system based on an improved image recognition model. First, a first image of the target tractor along its route is acquired by a first acquisition device, and second, a second image is acquired from electronic QR code screens on the rear or sides of the trailer by a second acquisition device. Both are considered valid (image) information. Further, a deep neural network-based image segmentation model is used to accurately extract the tractor's structural features and the trailer's cargo area features from the first image information. Simultaneously, based on the type and value of the structural features, an anomaly detection strategy and weight coefficients are matched from a pre-set strategy library using a rule engine. The cargo area features are input into this strategy, and an anomaly detection score is calculated using deduction items and weight coefficients, serving as the monitoring status information. In other words, this monitoring equipment early warning method and system no longer relies on the tractor's own computing power but instead uses external image acquisition devices and deep neural network models for data analysis, thereby reducing system costs and preventing monitoring failures due to transport equipment malfunctions. Furthermore, it reduces the amount of communication data and data processing, improving processing efficiency and enhancing real-time performance and reliability. By combining the first and second image information, the loading status of the target traction equipment can be analyzed more comprehensively, reducing the possibility of misjudgment and omission. In this application, "user" refers to dispatchers or managers responsible for monitoring and managing the transportation process, while "loading personnel" refers to staff responsible for loading cargo in the cargo loading area (using equipment), directly participating in the cargo loading process to ensure that the cargo is correctly loaded according to the preset loading plan.

[0032] The technical solutions of the embodiments of this application and how the technical solutions of the embodiments of this application solve the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the various embodiments or technical features described below can be arbitrarily combined to form new embodiments, and the same or similar concepts or processes may not be described again in some embodiments. Obviously, the described embodiments are some embodiments of the embodiments of this application, but not all embodiments.

[0033] Method implementation examples.

[0034] See Figure 1 , Figure 1 This is a flowchart illustrating a monitoring equipment early warning method provided in an embodiment of this application. This embodiment provides a monitoring equipment early warning method based on an improved image recognition model, the method comprising: S101, Use monitoring equipment to obtain effective image information of the target traction equipment; This can be understood as the monitoring equipment being an external acquisition device relative to the target traction equipment; the acquisition of effective image information does not rely on the traction vehicle's own sensors. By acquiring images through external devices, reliance on the traction vehicle's own computing power and sensors is avoided, thus reducing the hardware requirements on the target traction equipment.

[0035] S102, Based on the effective image information, the monitoring status information of the target traction device is obtained using an improved image recognition model; The improved image recognition model includes an image segmentation model and a detection and scoring model; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain monitoring status information based on the target features.

[0036] This can be understood as follows: In the improved image recognition model, the image segmentation model is used to perform pixel-level semantic segmentation of effective image information, extracting key features. These key features include, for example, the structural features of the tractor unit (such as vehicle body color and pattern features, vehicle body size features, and license plate number features) and the features of the cargo area in the trailer (such as the projection area of ​​the cargo on the tractor unit and the shape features of the cargo in the trailer's cargo area). Based on the segmentation results, a detection and scoring model is used to obtain monitoring status information, which may include scoring data. The image segmentation model can accurately locate key areas, while the detection and scoring model quantifies the degree of anomaly, enabling subsequent early warnings.

[0037] The extracted features are evaluated using a detection scoring model to generate an anomaly detection score, and the score is used to determine whether the cargo is abnormal.

[0038] Furthermore, the image segmentation model employs an encoder-decoder architecture. The encoder extracts features from valid image information, and the decoder outputs semantic segmentation results. An uncertainty estimation branch is added at the end of the decoder to calculate the uncertainty value of each pixel based on the softmax probability distribution. The uncertainty measure can be in the form of softmax entropy or softmax distance to quantify the reliability of the segmentation results. When the uncertainty measure exceeds its corresponding preset threshold, it indicates that the segmentation reliability of that region is low, possibly due to occlusion, motion blur, or uneven illumination. This region is marked as a high-uncertainty region, and the parameters of the first acquisition device are adjusted for re-acquisition, or the weight of the features of this region in the subsequent anomaly detection scoring is reduced.

[0039] S103, when the monitoring status information indicates that the load on the target traction device is abnormal, an early warning message is generated and sent to the user device. The user device may be a mobile phone, tablet, desktop computer, or industrial control computer, and the early warning message may be displayed to the user in the form of a text message, voice message, or pop-up window.

[0040] The technical solution provided in this embodiment does not rely on the computing power of the traction equipment itself, reducing the hardware requirements of the transportation device and thus lowering the overall monitoring and early warning costs. The image recognition model can quickly process image information, promptly detect anomalies and issue early warnings, and does not depend on the state of the transportation device itself. Compared with related technologies that rely on the transportation device itself for monitoring, which suffers from large data processing volumes and low efficiency, this enhances the real-time performance and reliability of monitoring.

[0041] See Figure 2 , Figure 2 This is a schematic diagram of a process for obtaining monitoring status information provided in an embodiment of this application.

[0042] In some embodiments, the target traction device includes a tractor and a trailer, and step S102 includes: S201, Based on the effective image information, extract the main structural features of the tractor and the cargo area features of the trailer using an image segmentation model and use them as target features in the effective image information; Features of the tractor's components are extracted using an image segmentation model as ontological structural features. Semantic segmentation is performed on the trailer area to extract cargo-related features, including cargo outline and stacking parameters. The tractor body and the trailer cargo are treated as independent segmentation targets to avoid mutual interference between their features. Target features from effective image information are used as structured output, that is, the segmentation results are converted into quantifiable parameters (such as coordinates, area, and angle) rather than raw pixel data, reducing the computational complexity of subsequent steps.

[0043] S202, Obtain an anomaly judgment strategy based on the body structure characteristics; Based on the anomaly judgment strategy and the characteristics of the carrying area, obtain an anomaly detection score using the detection scoring model, and use it as monitoring status information.

[0044] Obviously, compared with the two-step single-stage detection model in related technologies, the improved image recognition model mentioned in this application is a cascaded structure of "detection-re-detection" with the first and second detection models in the two-step single-stage detection model being executed sequentially. The purpose is to improve the accuracy of anomaly detection in a specific region (the region to be detected) so as to avoid missed detection. The image segmentation model and detection scoring model in this application are not simple cascaded detection models, but rather a combination of models with clearly defined functions and working collaboratively: the image segmentation model is responsible for extracting target features (such as ontological structural features and cargo area features) of the tractor and trailer from the original image. This is a feature extraction and localization process, and obviously, the output at this stage is not a detection result; the detection scoring model quantifies and scores based on the extracted features to generate monitoring status information. This is different from the scoring formula used in related technologies. This is because the application scenarios of related technologies are more focused on the detection of abnormal objects on roads in autonomous driving. Their models are single, preset abnormal detection models, and their scoring is based on the image region after superpixel segmentation. The purpose is to sharpen the abnormal image and reduce false positives. They mainly use superpixel segmentation technology to process the image, subdividing the digital image into multiple image sub-regions to achieve the detection of distant and smaller abnormal objects. This technology is suitable for the identification needs of small objects in the assisted driving scenarios they focus on, which is different from the needs of this application based on the actual factory tractor trailer cargo status monitoring and actual application scenarios. The feature extraction and rule scoring architecture protected in this application aims to combine the powerful feature extraction capabilities of deep learning models with the interpretability and flexibility of rule engines to address the complexity and dynamism of the specific scenario of monitoring the loading status of tractor trailers in factories.

[0045] Step S103 includes: when the anomaly detection score is not higher than a first threshold, it is considered that the load of the target traction device is abnormal, a first warning message is generated and sent to the user equipment. The anomaly detection score is, for example, a percentage system, such as 70, 80, or 90; the value range of the first threshold is, for example, (60, 70], such as 65; the value range of the second threshold mentioned below is, for example, (70, 75], such as 73.

[0046] Therefore, by dynamically selecting anomaly detection strategies based on the tractor's physical state, misjudgments caused by a one-size-fits-all approach are avoided. Phased feature processing extracts features of both the tractor and the cargo in the trailer independently, preventing feature extraction failures due to occlusion (such as cargo obscuring the license plate).

[0047] In some embodiments, the monitoring equipment includes a plurality of first acquisition devices disposed along the operating route of the target traction equipment and a plurality of second acquisition devices disposed in a plurality of cargo loading areas along the route; Step S101 includes: acquiring first image information of the target traction device along its travel route using a first acquisition device, and acquiring second image information of the target traction device in the cargo loading area using a second acquisition device; The first acquisition device is installed along the operating route of the target traction equipment to acquire initial image information of the traction equipment during its movement. This first acquisition device may be, for example, an image acquisition device such as a surveillance camera array, capable of capturing real-time images of the traction equipment's appearance and cargo loading status as it moves to the camera's location.

[0048] As an example, a pan-tilt control module and an ambient light sensor are integrated into the first acquisition device (surveillance camera array). A standard acquisition angle is determined based on the vehicle's structural characteristics (vehicle model). When the actual acquired image deviates from the standard angle by more than a threshold, the camera's posture is automatically adjusted; when changes in vehicle speed cause image blur, the exposure time and aperture size are automatically adjusted.

[0049] It can automatically adjust the acquisition parameters according to the type of traction equipment, vehicle speed, and lighting conditions; when it detects image blur, abnormal exposure, or incomplete target area, it triggers the camera adjustment strategy; it continuously acquires video of the cargo area and detects the frequency of cargo shaking (stability assessment).

[0050] Specifically, before acquiring valid image information, a standard acquisition angle is first determined based on the structural characteristics of the target towing equipment. The standard acquisition angle is the optimal shooting angle pre-calibrated for each specific vehicle model, including the camera's pitch angle, horizontal rotation angle, and focal length parameters. When the towing equipment passes a monitoring point, the corresponding standard acquisition angle parameters are invoked according to the vehicle model, driving the pan-tilt mechanism to adjust the camera's attitude.

[0051] During the acquisition process, the quality of the first image information is monitored in real time. When it is detected that the first image information does not contain a complete object area, or the image sharpness is lower than a preset threshold, or the exposure is abnormal, a first adjustment strategy for the camera is obtained. This first adjustment strategy is based on the deviation between the current image and the standard acquisition viewpoint, and includes adjusting one or more of the camera's position, orientation, focal length, and aperture size to ensure that the re-captured second image information meets the quality requirements.

[0052] Specifically, when the tractor's speed exceeds the preset speed, it is assumed that this will cause image motion blur, so the exposure time is shortened and the aperture is increased; when the cargo stacking height exceeds the expectation, resulting in the top area not being fully captured, the pitch angle is adjusted and the focal length is increased; when drastic changes in ambient light cause overexposure or underexposure, the aperture size is adjusted and adaptive gain control is enabled.

[0053] The second data acquisition device includes cameras positioned in multiple cargo loading areas along the travel route to acquire second image information of the traction equipment within these loading areas. It specifically targets cargo loading areas for image acquisition, meaning the route can be considered to include multiple cargo loading areas between the start and end points, used to sequentially load cargo into the trailer according to pre-set loading information in a pre-defined loading plan.

[0054] See Figure 3 , Figure 3 This is a partial flowchart illustrating the process of generating early warning information provided in an embodiment of this application.

[0055] Step S103 also includes: S301, when the anomaly detection score is between the first threshold and the second threshold, preset loading information is obtained based on the second image information; the second threshold is greater than the first threshold. S302, Based on the preset loading information and the current position information of the target traction device, obtain the loading area prediction information of the trailer; S303, based on the loading area prediction information and the first image information, determine whether the trailer is loaded incorrectly; when it is determined that the trailer is loaded incorrectly, generate a second warning message and send it to the user equipment.

[0056] When the anomaly detection score is not higher than the first threshold, a serious anomaly (such as cargo falling) is considered to have occurred, and a first warning message is generated and sent to the user equipment. When the score is between the first and second thresholds, considering that loading problems (such as incorrect quantity or type of cargo) are relatively common anomalies during the transportation process of tractor-trailers and trailers, loading problems are different from cargo falling, but they can still affect the stability of cargo during transportation. Preset loading information is obtained using second image information, and combined with the current location information, the predicted loading area information of the trailer is obtained. By comparing the predicted loading area information with the actual first image information, it is determined whether the trailer is incorrectly loaded (in the most recent previous loading area). If a loading error is determined, a second warning message is generated and sent to the user equipment.

[0057] Therefore, by setting up multiple data acquisition devices along the traction equipment's operating route and in the cargo loading area, comprehensive monitoring of the traction equipment at different locations and stages is achieved. This multi-point monitoring approach enables timely detection of anomalies during the traction equipment's movement and loading process, improving the comprehensiveness and accuracy of monitoring. Two thresholds are set, and different levels of early warning information are generated based on anomaly detection scores. This dual early warning mechanism allows for more flexible responses to anomalies of varying degrees. For example, when an anomaly is severe, a first early warning is generated promptly, reminding the user to take immediate action; when an anomaly is at a critical state, further analysis of loading information is conducted to determine if there are any loading errors, generating a second early warning, allowing the user to check the loading status in a timely manner. Through real-time monitoring and early warning of the traction equipment's cargo status, anomalies can be detected and handled promptly, reducing transportation accidents caused by cargo falling or improper loading, improving the safety of the transportation process, and reducing transportation risks and losses.

[0058] In some embodiments, the second image information is QR code image information; the method of obtaining the second image information of the target traction device in the cargo loading area includes: The second image information is obtained from the electronic QR code screen located at the tail or sides of the trailer using the second acquisition device.

[0059] The QR code displayed on the electronic QR code screen contains cargo loading information. The acquired QR code image information is decoded to obtain the cargo loading status after passing through the corresponding cargo loading area. The electronic QR code screen, for example, is an e-ink screen, including a screen and a communication module. The communication module communicates with a local server to obtain preset loading information sent by the warehouse management system running on the local server. The preset loading information is encoded into a QR code and displayed on the electronic QR code screen.

[0060] Therefore, by using QR code image information for monitoring, the reliance on the hardware of the traction equipment itself is reduced, thus lowering the hardware cost of the monitoring system. Equipping the trailer with an electronic QR code screen is relatively inexpensive and easy to integrate and promote.

[0061] In some embodiments, the step of obtaining an anomaly judgment strategy based on ontological structural features; and obtaining an anomaly detection score using the detection scoring model based on the anomaly judgment strategy and the features of the carrying area, and using it as monitoring status information, includes: Based on the type and value of the ontology structure features, the rule engine matches anomaly detection strategies and weight coefficients from the pre-set strategy library. The features of the carrying area are input into the anomaly judgment strategy, and the anomaly detection score is calculated according to the deduction items and weight coefficients; the anomaly detection score is used as monitoring status information.

[0062] As a rule-based scoring model, the detection scoring model is used to assess the degree of anomaly in the state of a carrying area. By matching rules and calculating scores, it can quickly determine whether a carrying object is abnormal, making it suitable for real-time monitoring and rapid decision-making. An example is the decision tree model, which divides the feature space of the object to be scored into multiple decision regions, and evaluates it according to corresponding rules within each region.

[0063] The ontological structural features include the shape of the vehicle's front end, body outline, color, and license plate number. The types and values ​​of these features will serve as the basis for matching anomaly detection strategies. From a pre-set strategy library, based on the types and values ​​of the ontological structural features, the anomaly detection strategy best suited to the current tractor unit and its corresponding weight coefficients are matched. It can be assumed that the pre-set strategy library stores multiple anomaly detection strategies for tractor units, each strategy corresponding to a specific range of ontological structural features and weight coefficients.

[0064] Deductions are for situations that may indicate abnormalities, such as irregular cargo shape, exceeding size limits, or misalignment. Weighting coefficients are adjusted based on changes in the cargo's structural features and the characteristics of the carrying area to reflect the importance of different features in the current situation.

[0065] An anomaly detection score is calculated by combining all deductions and weighting coefficients. This score will be used as monitoring status information to determine whether any anomalies have occurred in the cargo.

[0066] Therefore, the extracted features are evaluated and scored based on predefined rules and logic through a rule engine. The flexibility and interpretability of the rule engine allow anomaly detection strategies to be easily adjusted and optimized, enabling them to quickly adapt to different application scenarios and needs.

[0067] The formula corresponding to the detection scoring model can be:

[0068] Where S is the anomaly detection score, representing the final monitoring status information used to determine whether the load has experienced any anomalies. S0 is the initial score, representing the base score under the condition that no anomalies are found, usually set to a fixed value, such as 100 points. D i Let be the i-th deduction item, representing the deduction value for the i-th possible abnormal situation, determined based on the actual detection results. If this abnormal situation occurs, then D... i Take the corresponding deduction value; otherwise, it is 0. Wi is the i-th weight coefficient, representing the importance of the i-th deduction item in the overall score. It is dynamically adjusted according to the ontological structural features and the features of the carrying area. n is the total number of deduction items, representing a total of n possible abnormal situations that need to be evaluated.

[0069] In some embodiments, S301 further includes obtaining preset loading information based on the second image information when the anomaly detection score is between a first threshold and a second threshold; the second threshold is greater than the first threshold; and at the same time, continuous video acquisition is performed on the loading area to obtain its temporal stability characteristics. For example, optical flow or keypoint tracking algorithms can be used to analyze the pixel displacement of the load area of ​​the traction equipment between consecutive frames, and calculate the jitter characteristics (frequency) of the cargo within a predetermined time period. The jitter characteristics reflect the stability of the cargo during transportation; for example, an abnormally high jitter frequency indicates loose binding or an unstable center of gravity.

[0070] In this case, temporal stability features are fused with spatial features obtained from image segmentation to form the target features, which are then input into the detection scoring model. In addition to the aforementioned deduction items, the scoring formula of the detection scoring model includes a deduction item for temporal stability. When the jitter frequency exceeds a safety threshold, the deduction value is increased, reducing the anomaly detection score; when the jitter frequency is within the safety threshold range, no points are deducted.

[0071] In some embodiments, the detection scoring model is driven by a rule engine. A pre-built strategy library stores various anomaly detection strategies, each associated with a set of ontology structural features, transportation stages, and operating conditions. Transportation stages include the loading stage, the first to nth transportation stages, and the pre-unloading stage. Operating conditions include vehicle speed ranges and ambient light levels, etc.

[0072] Based on the current transportation task context, the optimal anomaly detection strategy and weighting coefficients can be matched from a pre-set strategy library. For example, during the loading phase, the focus is on the neatness of the cargo arrangement and the loading height; during the transportation phase, the focus is on cargo displacement and the integrity of the covering; on bumpy roads, the sensitivity to minor displacements is appropriately relaxed, but the detection weight for loose bindings is increased.

[0073] In some embodiments, considering that transportation tasks are pre-determined based on production plans and order requirements, the transportation routes of each tractor are usually fixed. Therefore, the transportation tasks performed by the same tractor over a period of time are repetitive and unchanging. It can be assumed that the pre-set strategy library stores anomaly detection strategies corresponding to multiple tractors (and their structural features). The advantage is that in practical applications, when production plans and order requirements change, the anomaly detection strategy for each tractor can be adjusted in real time by uniformly updating the data in the pre-set strategy library.

[0074] In some embodiments, the first threshold is: T1=S0 (1-α). α is the risk coefficient under severe abnormal conditions, for example, 0.3. The second threshold is the product of the first threshold and the threshold correlation coefficient. The threshold correlation coefficient is greater than 1, for example, 1.03 or 1.05.

[0075] In some embodiments, obtaining the predicted loading area information of the trailer based on the preset loading information and the current location information of the target traction device includes: The loading plan for the target traction equipment is obtained from the preset loading information, the loading plan including the loading sequence and placement requirements of the goods in different loading areas; Based on the loading plan and current location information, the previous loading area reached by the target traction equipment is taken as the target loading area; Based on the cargo information of the target loading area in the loading plan, predict the types, quantities, and arrangement of cargo in the trailer's loading area, and generate loading area prediction information.

[0076] As an example, the loading plan for the target traction equipment is extracted from preset loading information. The loading plan details the loading sequence and placement requirements of goods in different loading areas, including the loading area number and location, as well as the type, quantity, and arrangement of goods in each loading area, and the loading order. The current location information of the target traction equipment is obtained via GPS, including the specific coordinates of the traction equipment along the travel route and its trajectory.

[0077] The geographical range or coordinates of each loading area are predefined. The current position (travel trajectory) of the traction equipment is compared with the position of each loading area to find the matching previous loading area (i.e. the loading area where the goods were loaded most recently).

[0078] Based on the cargo information from the previous loading area in the loading plan, predict the type, quantity, and arrangement of cargo in the next loading area of ​​the trailer. Extract cargo information, including type, quantity, and arrangement, from the previous loading area in the loading plan, and then generate loading area prediction information to describe the expected loading situation of the trailer in that area. Integrate the predicted cargo type, quantity, and arrangement into the loading area prediction information.

[0079] Therefore, by acquiring the real-time location information of the traction equipment, the loading area to which the traction equipment has arrived can be dynamically determined. Loading area prediction information is only generated when the anomaly detection score is between the first and second thresholds, reducing data processing. This reduction in data processing allows for more efficient use of computing resources and lower energy consumption. The loading and unloading status of the trailer can be predicted in advance before entering the next loading area, helping users to identify potential loading problems early and reminding them to instruct loading personnel to check and handle issues in the next loading area, thus reducing transportation accidents caused by improper loading.

[0080] In some embodiments, determining whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information includes: The loading similarity between the predicted loading area information and the first image information is obtained. If the loading similarity value is less than a preset similarity value, the loading is considered to be incorrect; otherwise, the loading is considered to be normal, and statistics are started and the number of statistics is incremented by one. When the number of statistics exceeds a preset number, the number of statistics is reset to zero, and a second threshold anomaly information is sent to the user equipment. The second threshold anomaly information is used to prompt a reduction in the value of the second threshold. The preset number of times is, for example, 1, 2, or 3.

[0081] One method for obtaining loading similarity is to calculate the similarity by comparing the semantic consistency between the image information of the loading region prediction information and the first image information. This involves semantically labeling the objects in the image and then comparing the semantic consistency between the prediction information and the actual image.

[0082] Therefore, when the number of statistical counts exceeds a preset limit, the user is prompted to lower the second threshold. This helps users adjust parameters according to actual conditions, improving the adaptability and accuracy of the method. The alert is only triggered when the similarity falls below the threshold, reducing unnecessary data processing and improving the method's execution efficiency.

[0083] As an example, a weight of 0.04 is used for cases where the cargo size is out of range, and a weight of 0.02 is used for cases where the cargo position is off-center. Weighting coefficient W i The points can be dynamically adjusted based on the actual situation. For example, if a tractor unit is more sensitive to cargo position deviation (more prone to cargo position deviation), the weighting coefficient for that deduction item can be increased.

[0084] The image segmentation model is used to extract the structural features of the target equipment body at the front of the tractor and the features of the cargo area in the trailer from the first image information. It can be a deep learning-based convolutional neural network architecture with a U-Net structure. During training, a large amount of image data of the tractor under different scenarios is collected, including normal driving, cargo falling off, cargo shifting, etc., as training data. This embodiment does not limit parameters such as the learning rate (e.g., 0.001), the number of iterations (e.g., 500), and the batch size (e.g., 32).

[0085] As an example, a monitoring equipment early warning method based on an improved image recognition model is provided, applied to an early warning system in a factory environment. The early warning system includes monitoring equipment for acquiring effective image information of a target tractor. The monitoring equipment includes multiple first acquisition devices located along the operating route of the target tractor and multiple second acquisition devices located in multiple cargo loading areas along the route. By monitoring the loading status of the tractor's trailer using external image recognition technology, it can distinguish between serious cargo anomalies (such as falling cargo) and loading errors without relying on the computing power of the tractor's own equipment, reducing false alarms and improving interpretability and flexibility.

[0086] Specifically, the methods include: The first acquisition device is used to acquire first image information of the target traction equipment along its travel route, and the second acquisition device is used to acquire second image information from an electronic QR code screen located at the rear or sides of the trailer. The second image information is QR code image information. The target traction equipment includes a tractor and a trailer. Based on the valid image information, the image segmentation model extracts the main structural features of the tractor and the cargo area features of the trailer from the first image information and uses them as target features in the valid image information; based on the type and value of the main structural features, the rule engine matches the anomaly judgment strategy and weight coefficient from the pre-set strategy library; the cargo area features are input into the anomaly judgment strategy, and the anomaly detection score is calculated according to the deduction items and weight coefficients; the anomaly detection score is used as monitoring status information. When the anomaly detection score is not higher than the first threshold, the loading of the target traction device is considered abnormal, a first warning message is generated and sent to the user equipment; when the anomaly detection score is between the first and second thresholds, preset loading information is obtained based on the second image information; if the second threshold is greater than the first threshold, the loading plan of the target traction device is obtained from the preset loading information, which includes the loading order and placement requirements of goods in different loading areas; based on the loading plan and the current location information, the previous loading area reached by the target traction device is taken as the target loading area; based on the cargo information of the target loading area in the loading plan, the type, quantity and arrangement of cargo in the loading area of ​​the trailer are predicted, and loading area prediction information is generated; the loading similarity between the loading area prediction information and the first image information is obtained, and if the loading similarity value is less than the preset similarity, the loading is considered to be incorrect, and a second warning message is generated and sent to the user equipment when the trailer loading is determined to be incorrect; otherwise, the loading is considered to be normal, statistics are started and the statistics count is incremented by one, and when the statistics count is greater than the preset count, the statistics count is reset to zero, and a second threshold anomaly message is sent to the user equipment, which is used to prompt a reduction in the value of the second threshold.

[0087] Among them, the improved image recognition model includes an image segmentation model based on deep neural networks and a detection and scoring model driven by a rule engine; the image segmentation model is used to extract target features from effective image information, and the detection and scoring model is used to obtain monitoring status information based on target features.

[0088] See Figure 6 As another example, a monitoring device early warning method based on an improved image recognition model is provided, including: Acquire first image information by using a first acquisition device to acquire first image information of the target traction device along its travel route; the first acquisition device is a monitoring camera array. The second image information is obtained by using a second acquisition device to acquire the second image information from an electronic QR code screen located at the tail or sides of the trailer; the second image information is QR code image information; the second acquisition device is a QR code scanning device located on the top of the loading area. The main body structure features and the cargo area features are obtained. The main body structure features of the tractor and the cargo area features of the trailer are extracted based on the information of the first image using an image segmentation model. Obtain the anomaly detection strategy and weight coefficients. Based on the type and value of the ontology structure features, match the anomaly detection strategy and weight coefficients from the pre-set strategy library through the rule engine. The pre-set strategy library can be stored in relational databases such as MySQL and PostgreSQL. Calculate the anomaly detection score by inputting the features of the loading area into the anomaly judgment strategy, and calculate the anomaly detection score according to the deduction items and weight coefficients; use the anomaly detection score as monitoring status information. The score is compared with the first threshold. When the anomaly detection score is not higher than the first threshold, it is considered that the load of the target traction device is abnormal, a first warning message is generated and sent to the user equipment; when it is higher, it is determined whether it is between the first threshold and the second threshold. If the condition is met, the preset loading information is obtained based on the second image information; otherwise, no action is performed. Obtain the loading plan by retrieving the loading plan for the target traction equipment from the preset loading information. The loading plan includes the loading sequence and placement requirements of the goods in different loading areas. Obtain loading area prediction information. Based on the cargo information of the target loading area in the loading plan, predict the cargo type, quantity and arrangement of the cargo in the trailer's loading area, and obtain loading area prediction information. Among them, the previous loading area reached by the target traction equipment can be used as the target loading area based on the loading plan and the current location information. Obtain the loading similarity, which is the similarity between the predicted loading region information and the first image information; If the loading similarity value is less than the preset similarity value, a second warning message is generated and sent to the user device. If not, start the statistics and increment the count by one; If the number of counts exceeds the preset number, a second threshold anomaly message is generated and sent to the user device; otherwise, no action is taken.

[0089] It can be assumed that when the anomaly detection score is between the first and second thresholds (an intermediate state of suspected anomaly), an alarm is not triggered directly, but rather an analysis process is initiated: 1) Obtain preset loading information based on second image information (such as QR codes) from the cargo loading area; 2) Based on the current location information, predict the loading area where the trailer should be located and the condition of the cargo (loading area prediction information). 3) Compare the loading similarity between this predicted information and the first image information collected along the route; 4) Determine if there is an error in the loading based on the similarity, and generate a second warning message accordingly; 5) Introduce a statistical and feedback mechanism: If the loading is judged to be normal multiple times, prompt for adjustment of the second threshold. This series of steps constitutes a complete, closed-loop decision chain.

[0090] In summary, the technical solution provided in this example does not rely on the computing power and sensors of the traction equipment itself, but acquires and analyzes image information through externally deployed monitoring equipment (first and second acquisition devices). Specifically, it first uses the first and second acquisition devices to acquire valid image information; then, it extracts the structural features of the tractor and the cargo area features of the trailer from the first image information using an image segmentation model as target features; next, based on the type and value of the structural features, it matches the corresponding anomaly judgment strategy and weight coefficient from a pre-set strategy library; then, it inputs the cargo area features into the anomaly judgment strategy, calculates the anomaly detection score according to the deduction items and weight coefficients using a detection scoring model, and uses this score as monitoring status information; finally, when the anomaly detection score is not higher than a first threshold, it is considered that the cargo is abnormal, a first warning message is generated and sent to the user equipment; when the score is between the first and second thresholds, it uses the second image information to obtain preset loading information, and combines this with the current position information of the target traction equipment to obtain the predicted loading area information of the trailer, and then judges whether the trailer is incorrectly loaded based on this predicted information and the first image information. If so, a second warning message is generated and sent to the user equipment.

[0091] Therefore, by setting up multiple acquisition devices along the traction equipment's travel route and in the cargo loading area, comprehensive monitoring of the traction equipment at different locations and stages is achieved, improving the comprehensiveness and accuracy of monitoring. Utilizing an image segmentation model based on deep neural networks, the feature information of the tractor and trailer can be accurately extracted, providing a more accurate data foundation for subsequent anomaly detection and improving its precision. Through weighting coefficients, the anomaly detection score can more flexibly reflect the importance of different features in the current situation, improving the accuracy and adaptability of the score. Setting two thresholds generates different levels of early warning information based on the anomaly detection score; this dual early warning mechanism can more flexibly respond to different degrees of anomalies and promptly remind users to take action. Combining preset loading information, cargo retrieval information, and current location information, the loading area prediction information of the trailer is obtained. By comparing the predicted information with the actual image information, it intelligently determines whether the trailer is incorrectly loaded, providing users with a more accurate decision-making basis. Through real-time monitoring and early warning of the traction equipment's loading status, anomalies can be detected and handled promptly, reducing transportation accidents caused by cargo falling or improper loading, and improving the safety of the transportation process.

[0092] Equipment implementation example.

[0093] This application also provides a monitoring equipment early warning device based on an improved image recognition model. Its specific implementation and the achieved technical effects are consistent with those described in the above-mentioned method implementations, and some details will not be repeated. The monitoring equipment early warning device, the improved image recognition model, and the pre-set strategy library can be deployed on a cloud server, a local server, or through an edge-cloud collaborative architecture. In the edge-cloud collaborative architecture, the edge device can be an industrial computer or a high-performance embedded device, and the cloud server can be a virtual machine provided by a cloud service provider. This allows for preliminary image processing and anomaly detection on the edge device, while critical data is transmitted to the cloud server for further analysis and decision-making. In other words, the edge device is responsible for the initial processing of real-time data to ensure low-latency response; the cloud server is responsible for in-depth analysis of complex data, providing more accurate decision support.

[0094] See Figure 4 , Figure 4 This is a schematic diagram of a monitoring and early warning device provided in an embodiment of this application. The device includes: The information acquisition module is used to acquire effective image information of the target traction equipment using monitoring equipment; The model processing module is used to obtain the monitoring status information of the target traction device based on the effective image information and using an improved image recognition model. The information sending module is used to generate early warning information and send it to the user equipment when the monitoring status information indicates that the load of the target traction device is abnormal; The improved image recognition model includes an image segmentation model and a detection and scoring model; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain monitoring status information based on the target features.

[0095] In some embodiments, the target traction device includes a tractor and a trailer, and the model processing module includes: The feature extraction unit is used to extract the body structure features of the tractor and the cargo area features of the trailer through an image segmentation model based on the effective image information, and use them as target features in the effective image information. The scoring acquisition unit is used to acquire an anomaly judgment strategy based on the ontological structure features; based on the anomaly judgment strategy and the features of the loading area, it uses the detection scoring model to acquire an anomaly detection score, which is then used as monitoring status information. The information sending module includes: a first anomaly judgment unit, which is used to determine that the load of the target traction device is abnormal when the anomaly detection score is not higher than a first threshold, generate a first warning message and send it to the user equipment.

[0096] In some embodiments, the monitoring equipment includes a plurality of first acquisition devices disposed along the operating route of the target traction equipment and a plurality of second acquisition devices disposed in a plurality of cargo loading areas along the route; the information acquisition module includes: The first information acquisition unit is used to acquire first image information of the target traction device along its travel route using the first acquisition device; The second information acquisition unit is used to acquire second image information of the target traction device in the cargo loading area using the second acquisition device; The information sending module also includes: A preset loading information acquisition unit is used to acquire preset loading information based on the second image information when the anomaly detection score is between a first threshold and a second threshold; the second threshold is greater than the first threshold. The loading area prediction information acquisition unit is used to acquire the loading area prediction information of the trailer based on the preset loading information and the current position information of the target traction device. The second anomaly detection unit is used to determine whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information; when the trailer is incorrectly loaded, a second warning message is generated and sent to the user equipment.

[0097] In some embodiments, the second image information is QR code image information; the method of obtaining the second image information of the target traction device in the cargo loading area includes: using a second acquisition device to obtain the second image information from an electronic QR code screen located at the tail or sides of the trailer.

[0098] In some embodiments, the scoring acquisition unit includes: The matching subunit is used to match anomaly detection strategies and weight coefficients from a pre-set strategy library based on the type and value of ontology structural features through a rule engine. The scoring subunit is used to input the features of the carrying area into the anomaly judgment strategy, calculate the anomaly detection score according to the deduction items and weight coefficients, and use the anomaly detection score as monitoring status information.

[0099] In some embodiments, the loading region prediction information acquisition unit includes: The planning acquisition subunit is used to acquire the loading plan of the target traction equipment from the preset loading information. The loading plan includes the loading sequence and placement requirements of the goods in different loading areas. The area confirmation subunit is used to identify the previous loading area reached by the target traction device as the target loading area based on the loading plan and the current location information. The information generation subunit is used to predict the type, quantity, and arrangement of cargo in the loading area of ​​the trailer based on the cargo information of the target loading area in the loading plan, and generate loading area prediction information.

[0100] In some embodiments, the second anomaly judgment unit is used to obtain the loading similarity between the loading region prediction information and the first image information. When the value of the loading similarity is less than a preset similarity, it is considered that the loading is wrong; otherwise, it is considered that the loading is normal, and it starts to count and increments the count by one. When the count is greater than the preset count, it resets the count to zero and sends a second threshold anomaly information to the user equipment. The second threshold anomaly information is used to prompt the user to reduce the value of the second threshold.

[0101] System Implementation Example.

[0102] See Figure 5 , Figure 5 This is a schematic diagram of a module of an early warning system provided in an embodiment of this application.

[0103] This application provides an early warning system, the specific implementation of which is consistent with the implementation method and the technical effects achieved in the above-described method implementation, and some contents will not be repeated.

[0104] The system includes the monitoring and early warning equipment described in the equipment embodiment, as well as monitoring equipment for acquiring effective image information of the target traction equipment.

[0105] In some embodiments, the monitoring equipment includes a plurality of first acquisition devices disposed along the operating route of the target traction equipment and a plurality of second acquisition devices disposed in a plurality of cargo loading areas along the route.

[0106] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more".

[0107] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are configured to distinguish similar objects and are not necessarily configured to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0108] This application describes the invention from the perspectives of purpose, performance, progress, and novelty, and it meets the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

Claims

1. A monitoring equipment early warning method based on an improved image recognition model, characterized in that, The method includes the following steps: S1, Use monitoring equipment to obtain effective image information of the target traction equipment; S2, based on the effective image information, use an improved image recognition model to obtain an anomaly detection score for the target traction device and use it as monitoring status information; the improved image recognition model is deployed on a cloud server, or on a local server, or through an edge cloud collaborative architecture; S3, based on the anomaly detection score, determine whether the load of the target traction device is abnormal, and generate an early warning message and send it to the user equipment when an anomaly occurs; The improved image recognition model includes an image segmentation model based on a deep neural network and a detection and scoring model driven by a rule engine; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain an anomaly detection score based on the target features.

2. The monitoring equipment early warning method according to claim 1, characterized in that, The target traction equipment includes a tractor and a trailer, and step S2 includes: Based on the effective image information, the main structural features of the tractor and the cargo area features of the trailer are extracted by the image segmentation model and used as target features in the effective image information; An anomaly detection strategy is obtained based on the characteristics of the body structure; based on the anomaly detection strategy and the characteristics of the loading area, an anomaly detection score is obtained using the detection scoring model and used as monitoring status information; Step S3 includes: when the anomaly detection score is not higher than the first threshold, it is considered that the load of the target traction device is abnormal, a first warning message is generated and sent to the user device.

3. The monitoring equipment early warning method according to claim 2, characterized in that, The monitoring equipment includes multiple first acquisition devices installed along the operating route of the target traction equipment and multiple second acquisition devices installed in multiple cargo loading areas along the route. Step S1 includes: using a first acquisition device to acquire first image information of the target traction device along its travel route, and using a second acquisition device to acquire second image information of the target traction device in the cargo loading area; Step S3 further includes: when the anomaly detection score is between a first threshold and a second threshold, obtaining preset loading information based on the second image information; when the second threshold is greater than the first threshold; obtaining loading area prediction information for the trailer based on the preset loading information and the current position information of the target traction device; determining whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information; when it is determined that the trailer is incorrectly loaded, generating a second warning message and sending it to the user device; or, Step S3 further includes: obtaining preset loading information based on the second image information; the second threshold being greater than the first threshold; simultaneously, continuously acquiring video of the loading area to obtain its temporal stability features; and fusing the temporal stability features with the spatial features obtained from the image segmentation model as the target features.

4. The monitoring equipment early warning method according to claim 3, characterized in that, The second image information is QR code image information; the method of obtaining the second image information of the target traction device in the cargo loading area includes: using a second acquisition device to obtain the second image information from an electronic QR code screen set at the tail or sides of the trailer.

5. The monitoring equipment early warning method according to claim 3, characterized in that, The step of obtaining the predicted loading area information of the trailer based on the preset loading information and the current position information of the target traction device includes: The loading plan for the target traction equipment is obtained from the preset loading information, the loading plan including the loading sequence and placement requirements of the goods in different loading areas; Based on the loading plan and current location information, the previous loading area reached by the target traction equipment is taken as the target loading area; Based on the cargo information of the target loading area in the loading plan, predict the types, quantities, and arrangement of cargo in the trailer's loading area, and generate loading area prediction information.

6. The monitoring equipment early warning method according to claim 3, characterized in that, The step of determining whether the trailer is incorrectly loaded based on the loading area prediction information and the first image information includes: The loading similarity between the predicted loading area information and the first image information is obtained. When the loading similarity value is less than the preset similarity value, the loading is considered to be incorrect; otherwise, the loading is considered to be normal, and the counting begins and the counting count is incremented by one. When the counting count is greater than the preset count, the counting count is reset to zero, and a second threshold anomaly information is sent to the user equipment. The second threshold anomaly information is used to prompt the user to lower the value of the second threshold.

7. The monitoring equipment early warning method according to claim 2, characterized in that, The strategy for obtaining anomaly judgment based on ontological structural features; Based on the anomaly detection strategy and the characteristics of the loading area, the anomaly detection score is obtained using the aforementioned detection scoring model and used as monitoring status information, including: Based on the type and value of the ontology structure features, the rule engine matches anomaly detection strategies and weight coefficients from a pre-set strategy library. The features of the carrying area are input into the anomaly judgment strategy, and the anomaly detection score is calculated according to the deduction items and weight coefficients; the anomaly detection score is used as monitoring status information.

8. A monitoring and early warning device based on an improved image recognition model, characterized in that, include: The information acquisition module is used to acquire effective image information of the target traction equipment using monitoring equipment; The model processing module is used to obtain the monitoring status information of the target traction device based on the effective image information and using an improved image recognition model. The information sending module is used to generate early warning information and send it to the user equipment when the monitoring status information indicates that the load of the target traction device is abnormal; The improved image recognition model includes an image segmentation model and a detection and scoring model; the image segmentation model is used to extract target features from the effective image information, and the detection and scoring model is used to obtain monitoring status information based on the target features.

9. An early warning system, characterized in that, It includes the monitoring and early warning device as described in claim 8, and the monitoring device for acquiring effective image information of the target traction device.

10. The early warning system according to claim 9, characterized in that, The monitoring equipment includes multiple first acquisition devices located along the operating route of the target traction equipment and multiple second acquisition devices located in multiple cargo loading areas along the route.

Images