Processing method and device for illegal transportation of freight vehicle

Abstract

The invention provides a processing method and device for illegal transportation of a freight vehicle. The processing method comprises the steps of obtaining chat information of a driver side and a client side based on an order and image information of the freight vehicle in a transportation process; inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding violation text information; inputting the image information into an image recognition model for image recognition to obtain corresponding violation picture information; and based on the violation character information and the violation picture information, judging whether the order is transported in a violation manner or not. By means of the processing method and device for illegal transportation of the freight vehicle, whether the freight vehicle has illegal transportation or not can be rapidly judged.

Description

Technical Field

[0001] The present invention relates to the field of logistics, and in particular to a method and device for handling illegal transportation of freight vehicles. Background Art

[0002] In the field of freight logistics, drivers often violate relevant laws and regulations by using freight vehicles to illegally carry passengers, carry people in cargo boxes, mix people and goods, and only carry people without goods, driven by profit or other reasons. These violations not only increase transportation risks, but may also cause serious safety accidents and pose a threat to public safety. Therefore, freight logistics companies need to take effective monitoring measures to ensure the legality and compliance of the transportation process.

[0003] Traditional monitoring methods mainly involve installing cameras on freight vehicles and having backstage personnel conduct real-time review of videos and photos. However, real-time review requires a large amount of manual resources, especially when there are a large number of transport vehicles and complex transport routes. The number of backstage personnel required increases dramatically, resulting in excessive labor costs. Manual review of videos and images is inefficient, and violations can often only be discovered after the fact, making it difficult to stop and deal with them in a timely manner, increasing the possibility of accidents. Therefore, there is room for improvement. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for handling illegal transportation of freight vehicles, which can quickly determine whether the freight vehicles have engaged in illegal transportation.

[0005] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0006] The present invention provides a method for handling illegal transportation of freight vehicles, comprising:

[0007] Obtain chat information between the driver and the client based on the order, as well as image information of the freight vehicle during transportation;

[0008] Inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information;

[0009] Inputting the image information into an image recognition model for image recognition to obtain corresponding illegal picture information;

[0010] Based on the illegal text information and / or illegal picture information, determine whether the order has any illegal transportation.

[0011] In one embodiment of the present invention, the chat information includes voice call information and text chat information; the image information includes cargo box photo information and real-time video information; the step of inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information, and inputting the image information into an image recognition model for image recognition to obtain corresponding illegal picture information includes:

[0012] Convert the voice call information into text to generate corresponding call text information;

[0013] Inputting the call text information and the text chat information into a semantic recognition model for semantic recognition respectively, and obtaining the illegal text information in the call text information and its corresponding call violation confidence, and the illegal text information in the text chat information and its corresponding chat violation confidence;

[0014] The cargo box photographic information and the real-time video information are respectively input into an image recognition model for image recognition, so as to obtain the illegal picture information in the cargo box photographic information and its corresponding confidence level of the photographic violation, and the illegal picture information in the real-time video information and its corresponding confidence level of the video violation.

[0015] In one embodiment of the present invention, the step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes:

[0016] According to the illegal text information in the call text information and its corresponding call violation confidence level, the corresponding illegal behavior is determined:

[0017] When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0018] When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

[0019] In one embodiment of the present invention, the step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes:

[0020] According to the illegal text information in the text chat information and its corresponding chat violation confidence, the corresponding illegal behavior is judged:

[0021] When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0022] When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

[0023] In one embodiment of the present invention, the step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes:

[0024] According to the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, the corresponding illegal behavior is determined:

[0025] When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0026] When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

[0027] In one embodiment of the present invention, the step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes:

[0028] According to the illegal image information in the real-time video information and its corresponding video violation confidence, the corresponding illegal behavior is judged:

[0029] When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0030] When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

[0031] In one embodiment of the present invention, the step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes:

[0032] According to the illegal text information in the call text information and its corresponding call violation confidence, the illegal text information in the text chat information and its corresponding chat violation confidence, the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, and the illegal picture information in the real-time video information and its corresponding video violation confidence, the corresponding illegal behavior is determined:

[0033] When it is determined that the violation is that the driver agrees to illegal transportation, the corresponding violation information is fed back to the driver side and the client side.

[0034] In one embodiment of the present invention, after the step of feeding back to the driver the information of canceling the order without liability when judging that the illegal behavior is that the customer wishes to transport in violation of regulations, the step further includes:

[0035] Determine whether the driver cancels the order:

[0036] If so, ending the order;

[0037] Otherwise, the order will be reviewed and the driver will be punished based on the review results.

[0038] In one embodiment of the present invention, before the step of obtaining the chat information between the driver and the client based on the order and the image information of the freight vehicle during the transportation process, it also includes:

[0039] Get the client's order-based order note information;

[0040] Inputting the order note information into a semantic recognition model for semantic recognition, and obtaining illegal text information in the order note information and its corresponding note violation confidence;

[0041] According to the violation text information and its corresponding remark violation confidence, the corresponding violation behavior is determined:

[0042] When it is determined that the illegal behavior is that the customer wishes to transport in violation of regulations, the corresponding illegal information is fed back to the client.

[0043] The present invention also provides a device for processing illegal transportation of freight vehicles, comprising:

[0044] The information acquisition module is used to obtain the chat information between the driver and the client based on the order and the image information of the freight vehicle during the transportation process;

[0045] A semantic processing module, used for inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information;

[0046] An image processing module, used for inputting the image information into an image recognition model for image recognition to obtain corresponding illegal picture information; and

[0047] The violation judgment module is used to determine whether there is any illegal transportation in the order based on the illegal text information and the illegal picture information.

[0048] As described above, the present invention provides a method and device for handling illegal transportation by freight vehicles. By combining multiple information source identification technologies and formulating different disposal strategies for different stages (before, during, and after the trip), the platform can more accurately and timely identify and handle illegal loading of people in cargo boxes, which not only improves the accuracy of identification and reduces the possibility of false alarms, but also effectively protects the personal safety of drivers and users, and improves the platform's regulatory efficiency and user trust.

[0049] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0051] Figure 1 A flowchart of a method for handling illegal transportation by freight vehicles in one embodiment of the present invention;

[0052] Figure 2 It is a schematic diagram of a device for handling illegal transportation of freight vehicles in one embodiment of the present invention.

[0053] In the figure: 100, pre-processing module; 200, information acquisition module; 300, semantic processing module; 400, image processing module; 500, violation judgment module. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] See also Figure 1 The present invention discloses a method for handling illegal transportation of freight vehicles, which can monitor freight vehicles before and during transportation to prevent the occurrence of illegal transportation of freight vehicles. The method may include the following steps:

[0056] Step S10: Obtain order note information based on the order from the client;

[0057] Step S20: input the order note information into a semantic recognition model for semantic recognition, and obtain the illegal text information in the order note information and its corresponding note violation confidence;

[0058] Step S30: According to the illegal text information and its corresponding remark violation confidence, the corresponding illegal behavior is determined:

[0059] Step S40: when it is determined that the violation is that the customer wishes to transport in violation of regulations, corresponding violation information is fed back to the client;

[0060] Step S50: acquiring chat information between the driver and the client based on the order and image information of the freight vehicle during transportation;

[0061] Step S60: Input the chat information into the semantic recognition model for semantic recognition to obtain corresponding illegal text information; input the image information into the image recognition model for image recognition to obtain corresponding illegal picture information;

[0062] Step S70: Based on the illegal text information and / or illegal picture information, determine whether the order has any illegal transportation.

[0063] In some embodiments, when executing step S10, specifically, the order note information refers to the additional information entered by the user through the client before placing an order, which generally includes detailed descriptions of the type, quantity, volume, weight, etc. of the goods, as well as other special requirements or precautions. After receiving the order, the platform can preliminarily understand the goods of the order through the order note information, providing an important reference for subsequent review and management.

[0064] In some embodiments, when executing step S20, specifically, a semantic recognition model can be used to analyze the order note information to identify whether there is any illegal loading behavior in the cargo box. Through natural language processing (NLP) technology, this method can automatically detect illegal content in the order note and provide a violation confidence index to evaluate the credibility of the recognition result.

[0065] In some embodiments, the order note information can be preprocessed to generate corresponding word sequence information. Preprocessing can include text cleaning, word segmentation, stop word removal, and word vectorization. Among them, text cleaning refers to removing punctuation, numbers, special characters, etc. in the order note information, retaining the pure text content. Word segmentation refers to dividing the cleaned text into individual words or phrases to generate a word sequence. For example, the sentence "Please driver must not carry people, only transport goods" is divided into a word sequence: ["please", "driver", "must", "don't", "carry people", "only", "transport", "goods"]. Stop word removal refers to words that appear frequently in the text but contribute little to semantics, such as "please", "of", "yes", "yes", etc. Removing these stop words can reduce noise and improve the recognition efficiency and accuracy of the model. Word vectorization refers to converting each word in the word sequence into a numerical vector for deep learning model processing. Word embedding technology (such as Word2Vec, GloVe or BERT's word vector) can be used for conversion. After word vectorization processing, the corresponding word sequence information can be generated, such as ["please", "driver", "must", "don't", "passenger", "only", "transport", "cargo"], and the word vector sequence is expressed as [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9], [0.3, 0.4, 0.5], [0.6, 0.7, 0.8], [0.9, 0.1, 0.2], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9]].

[0066] In some embodiments, the word sequence information can then be input into a semantic recognition model for processing to generate illegal text information in the order note information and its corresponding note violation confidence. For example, a deep learning model in the field of NLP can be selected, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a Transformer, etc. Public large language models such as BERT, GPT, etc. can also be used. Specifically, the word vector sequence generated by preprocessing can be input into the semantic recognition model. The model extracts features from the word vector sequence through a multi-layer neural network to generate a high-level semantic representation. The model classifies and judges the generated semantic representation through the output layer, and outputs a note violation confidence A for illegal transportation, indicating whether there is content of illegal carrying of people in the order note. The note violation confidence for illegal transportation is a value between 0 and 1, indicating the degree of credibility of whether the cargo box is illegally loaded with people in the current algorithm model recognition result. The larger the note violation confidence, the higher the possibility of illegal loading and the higher the credibility.

[0067] In some embodiments, when step S30 is executed, specifically, in the freight logistics monitoring system, the semantic recognition model is used to identify whether there is content in the order note information that the cargo box is illegally loaded with people, and a note violation confidence A is generated. In order to decide whether to accept or reject the user's order, the violation confidence A can be compared with a predetermined note violation confidence threshold a. The size of the note violation confidence threshold a may be unlimited, for example, it may be 0.7 / 0.8 / 0.9, etc.

[0068] In some embodiments, when executing step S40, specifically, when the violation confidence A is greater than the preset remark violation confidence threshold a (A>a), the user's order is directly rejected. At this time, a prompt message can be sent to the user to clearly inform him / her of the reason why the order was rejected, for example: "Your order remarks mentioned that the cargo box was illegally loaded with people, and the system automatically rejected this order. Please ensure that your cargo transportation complies with relevant laws and regulations." The violation confidence A of the identification result is less than or equal to the preset remark violation confidence threshold a (A≤a), and the system allows the user to place an order. At this point, the order is successfully created and the process enters the next step. The system can further perform subsequent verification through other monitoring means.

[0069] In some embodiments, when executing step S50, specifically, in order to more accurately identify and manage violations during the freight process, this embodiment may adopt a multi-level information collection and verification method. For example, the acquired voice call information, text chat information, cargo box photo information, and real-time video information may be verified.

[0070] In some embodiments, voice call information refers to the content of the call when the driver receives the order and contacts the user by phone after the user completes the order. Through the phone call, the driver can further confirm the details of the goods, including the type, volume, weight, loading and unloading location, etc. The platform can record and store such voice call information.

[0071] In some embodiments, text chat information refers to the communication records between the driver and the user through instant messaging (such as IM) tools. These records usually include detailed descriptions of the goods, the specific time and place of loading and unloading, and the negotiation content of both parties. The platform can monitor and store these text chat information in real time.

[0072] In some embodiments, cargo box photo information refers to the picture information that the driver takes pictures of the goods in the cargo box according to the platform's requirements after loading is completed and uploads to the platform. These pictures can intuitively show the goods in the cargo box. The platform can monitor and store these cargo box photo information in real time.

[0073] In some embodiments, real-time video information refers to the continuous video recorded by the camera installed in the cargo box when the vehicle is loaded and on the way to the destination. These videos can monitor the dynamic situation in the cargo box in real time. The platform can monitor and store these real-time video information in real time.

[0074] In some embodiments, when executing step S60, specifically, the chat information includes voice call information and text chat information; the image information includes cargo box photo information and real-time video information. Step S60 may include the following steps:

[0075] Step S61, converting the voice call information into text to generate corresponding call text information;

[0076] Step S62: inputting the call text information and the text chat information into the semantic recognition model for semantic recognition, and obtaining the illegal text information in the call text information and its corresponding call violation confidence, and the illegal text information in the text chat information and its corresponding chat violation confidence;

[0077] Step S63: input the cargo box photo information and the real-time video information into the image recognition model for image recognition, and obtain the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, and the illegal picture information in the real-time video information and its corresponding video violation confidence.

[0078] In some embodiments, when executing step S61, specifically, the semantic recognition model can be used to identify whether the voice call information contains the content "the driver agrees to illegally carry passengers in the cargo box". For example, an open source or commercial automatic speech recognition (ASR) model can be used to convert the voice call information into text content and generate call text information.

[0079] In some embodiments, when executing step S62, specifically, the call text information can be input into a trained NLP deep learning model or semantic recognition model to obtain the violation text information in the call text information and its corresponding call violation confidence, for example, to identify whether it contains the content "the driver agrees to the illegal loading of people in the cargo box" and its corresponding call violation confidence. Alternatively, the speech recognition model can be directly used to directly identify the content "the driver agrees to the illegal loading of people in the cargo box" from the recording, skipping the text conversion step. Finally, the recognition result can be output, and the recognition result includes whether the driver agrees to the illegal loading of people in the cargo box and the call violation confidence B. B can be in the range of 0 to 1, and the larger the value, the higher the credibility of the recognition result.

[0080] In some embodiments, for example, a user communicates with a driver over the phone, and the content is "User: Can you give me a ride? Driver: No problem, I can help." Subsequently, the ASR model can be used to convert the phone recording into text, for example, "User: Can you give me a ride? Driver: No problem, I can help." Finally, the converted text content is input into a trained NLP model. The NLP model extracts semantic features of the text content. The NLP model outputs recognition results, including "The driver agrees to the illegal carrying of passengers in the cargo box" and "the corresponding call violation confidence level B."

[0081] In some embodiments, a semantic recognition model can be used to identify whether there is relevant information in the text chat message that the driver agrees to illegally carry passengers. For example, the text chat message can be input into a trained NLP deep learning model to obtain the illegal text information in the text chat message and its corresponding chat violation confidence. For example, it is identified whether the content "the driver agrees to illegally carry passengers in the cargo box" is included and its corresponding chat violation confidence. The recognition result may include whether the driver agrees to illegally carry passengers in the cargo box and the chat violation confidence C. C can be in the range of 0 to 1. The larger the value, the higher the credibility of the recognition result.

[0082] In some embodiments, for example, a user and a driver chat through an IM tool, and the content is "User: Can you help me pick up someone? Driver: OK, I will help." Subsequently, the chat content can be directly input into a trained NLP model. The NLP model extracts semantic features of the text content. The NLP model outputs a recognition result, including "The driver agrees to the illegal loading of passengers in the cargo box" and "the corresponding chat violation confidence C".

[0083] In some embodiments, when step S63 is executed, specifically, the image recognition model can be used to identify whether the cargo box photo information contains content that indicates illegal passenger transport in the cargo box. For example, the cargo box photo information is input into a trained image deep learning model to obtain illegal picture information in the cargo box photo information and its corresponding confidence level for illegal passenger transport. For example, it is determined whether the picture contains content that indicates illegal passenger transport and its corresponding confidence level for illegal passenger transport. The recognition result may include whether the cargo box is illegally carrying passengers and the confidence level D for illegal passenger transport. D may be in the range of 0 to 1, and the larger the value, the higher the credibility of the recognition result. Among them, the type of image recognition model may not be limited, for example, it may be DETR, ViT, YOLO7, etc.

[0084] In some embodiments, for example, after the driver has completed loading, he takes a photo of the cargo box and uploads it to the platform. Subsequently, the taken photo can be input into a trained image deep learning model. The model extracts visual features in the photo. The model outputs a recognition result, including "whether the cargo box is illegally loaded with people" and "the corresponding photo violation confidence D".

[0085] In some embodiments, an image recognition model can be used to identify whether there is any content in the real-time video information of the cargo box that contains illegal passenger transportation. For example, a camera can be installed in the cargo box to record in real time, and the video information can be input into a trained video deep learning model to obtain illegal image information in the real-time video information and its corresponding video violation confidence. For example, it can be identified whether the video contains content that illegally carries passengers and its corresponding video violation confidence. The recognition result can include whether the cargo box is illegally carrying passengers and the video violation confidence E. E can be in the range of 0 to 1. The larger the value, the higher the credibility of the recognition result.

[0086] In some embodiments, for example, a camera is installed in the cargo box to record real-time video while the vehicle is on its way to the destination. Subsequently, the real-time video information can be input into a trained video deep learning model. The model extracts visual features in the video. The model outputs recognition results, including "whether the cargo box is illegally carrying people" and "the corresponding video violation confidence E".

[0087] In some embodiments, when step S70 is executed, specifically, step S70 may include the following steps:

[0088] Step S711: According to the illegal text information in the call text information and its corresponding call illegal confidence level, the corresponding illegal behavior is determined:

[0089] Step S712: When it is determined that the violation is that the driver agrees to transport in violation of regulations, the corresponding violation information is fed back to the driver terminal and the client terminal;

[0090] Step S713: When it is determined that the violation is that the customer wishes to transport in violation of regulations, the driver is fed back information about the order being cancelled without liability.

[0091] In some embodiments, when step S711 is executed, specifically, in the freight logistics monitoring system, multiple steps (including voice call recognition, text chat recognition, cargo box photo recognition, and real-time video recognition) can be used to determine whether there is any illegal loading of people in the cargo box. The recognition result of each step has a corresponding violation confidence (B, C, D, E), which needs to be compared with the corresponding thresholds (b1, c1, d1, e1). If any violation confidence is greater than its corresponding threshold, the system will directly cancel the order and prompt the driver and user not to illegally load people in the cargo box. Among them, the specific size of the threshold (b1, c1, d1, e1) can be set according to actual needs.

[0092] In some embodiments, when executing step S712, specifically, the first call violation confidence threshold b1 can be used to determine the voice call violation confidence B. If the call violation confidence B is greater than b1, it means that there is illegal passenger carrying content with high violation confidence in the voice call information. At this time, the system will cancel the order immediately. The system sends a prompt message to the driver and the user, clearly informing them of the reason for the cancellation of the order, for example: "There is illegal passenger carrying behavior in the cargo box with high violation confidence in your voice call, and the system automatically cancels this order. Please ensure that your cargo transportation complies with relevant laws and regulations."

[0093] In some embodiments, when step S713 is executed, specifically, when the call violation confidence B ≤ the first call violation confidence threshold b1 and B> the second call violation confidence threshold b2, it is determined that the client wants to transport in violation of regulations, and feedback is given to the driver to cancel the order without responsibility. The second call violation confidence threshold b2 can be divided into a second call violation confidence upper threshold b2.1 and a second call violation confidence threshold b2.

[0094] In some embodiments, when the call violation confidence B ≤ the first call violation confidence threshold b1 and B> the second call violation confidence upper threshold b2.1, the specific processing may be a pop-up window reminder, and a pop-up window reminder on the driver side, "If the user wants the cargo box to carry passengers illegally, the order can be cancelled without liability." The driver can click a button in the pop-up window to cancel the order directly.

[0095] In some embodiments, when the call violation confidence B ≤ the second call violation confidence upper threshold b2.1 and B> the second call violation confidence threshold b2, the specific processing may be a voice broadcast reminder, and the driver may be reminded through voice broadcast that "the cargo box cannot be illegally loaded with people, and the order can be cancelled without liability". The driver may determine whether to cancel the order based on the voice reminder.

[0096] In some embodiments, when the call violation confidence B is less than or equal to the second call violation confidence threshold b2, or the driver has not cancelled the order, it can be determined that the driver has not violated the transportation regulations. The specific values ​​of the thresholds (b2.1, b2) can be set according to actual needs.

[0097] In some embodiments, when executing step S70, specifically, step S70 may further include the following steps:

[0098] Step S721: According to the illegal text information in the text chat information and its corresponding chat violation confidence, the corresponding illegal behavior is determined:

[0099] Step S722: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0100] Step S723: When it is determined that the violation is that the customer wishes to transport in violation of regulations, the driver is fed back information about the cancellation of the order without liability.

[0101] In some embodiments, when executing step S721 and step S722, specifically, the first chat violation confidence threshold c1 can be used to determine the text chat violation confidence C. If the chat violation confidence C is greater than c1, it means that there is illegal passenger carrying content with high violation confidence in the text chat information. At this time, the system will cancel the order immediately. The system sends a prompt message to the driver and the user, clearly informing them of the reason for the cancellation of the order, for example: "There is illegal passenger carrying behavior in the cargo box with high violation confidence in your text chat, and the system automatically cancels this order. Please ensure that your cargo transportation complies with relevant laws and regulations."

[0102] In some embodiments, when step S723 is executed, specifically, when the chat violation confidence C ≤ the first chat violation confidence threshold c1 and C> the second chat violation confidence threshold c2, it is determined that the client wants to transport in violation of regulations, and feedback is given to the driver to cancel the order without responsibility. The second chat violation confidence threshold c2 can be divided into a second chat confidence upper threshold c2.1 and a second chat violation confidence threshold c2.

[0103] In some embodiments, when the chat violation confidence C ≤ the first chat violation confidence threshold c1 and C> the second chat confidence upper threshold c2.1, the specific processing may be a pop-up window reminder, and a pop-up window reminder on the driver side, "If the user wants the cargo box to carry passengers illegally, the order can be cancelled without liability." The driver can click a button in the pop-up window to cancel the order directly.

[0104] In some embodiments, when the chat violation confidence C ≤ the second chat violation confidence upper threshold c2.1 and C> the second chat violation confidence threshold c2, the specific processing may be a voice broadcast reminder, and the driver may be reminded through voice broadcast that "the cargo box cannot be illegally loaded with people, and the order can be cancelled without liability". The driver may determine whether to cancel the order based on the voice reminder.

[0105] In some embodiments, when the chat violation confidence C is less than or equal to the second chat violation confidence threshold c2, or the driver has not cancelled the order, it can be determined that the driver has not violated the transportation regulations. The specific values ​​of the thresholds (c2.1, c2) can be set according to actual needs.

[0106] In some embodiments, when executing step S70, specifically, step S70 may further include the following steps:

[0107] Step S731: According to the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, the corresponding illegal behavior is determined:

[0108] Step S732: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0109] Step S733: When it is determined that the violation is that the customer wishes to transport in violation of regulations, the driver is fed back information about the order cancellation without liability.

[0110] In some embodiments, when executing step S731 and step S732, specifically, the first photo violation confidence threshold d1 can be used to determine the photo violation confidence D. If the photo violation confidence D is greater than d1, it means that there is illegal passenger carrying content with high violation confidence in the photo information. At this time, the system will cancel the order immediately. The system sends a prompt message to the driver and the user, clearly informing them of the reason for the cancellation of the order, for example: "There is illegal passenger carrying behavior in the cargo box with high violation confidence in your photo, and the system automatically cancels this order. Please ensure that your cargo transportation complies with relevant laws and regulations."

[0111] In some embodiments, when step S733 is executed, specifically, when the photo violation confidence D ≤ the first photo violation confidence threshold d1 and D> the second photo violation confidence threshold d2, it is determined that the client wants to transport in violation of regulations, and feedback is given to the driver to cancel the order without responsibility. The second photo violation confidence threshold d2 can be divided into a second photo violation confidence upper threshold d2.1 and a second photo violation confidence threshold d2.

[0112] In some embodiments, when the photo violation confidence D ≤ the first photo violation confidence threshold d1 and D> the second photo violation confidence upper threshold d2.1, the specific processing may be a pop-up window reminder, and a pop-up window reminder on the driver side, "If the user wants the cargo box to carry people illegally, the order can be cancelled without liability." The driver can click a button in the pop-up window to cancel the order directly.

[0113] In some embodiments, when the photo violation confidence D ≤ the second photo violation confidence upper threshold d2.1 and d> the second photo violation confidence threshold d2, the specific processing may be a voice broadcast reminder, and the driver may be reminded through voice broadcast that "the cargo box cannot be illegally loaded with people, and the order can be cancelled without liability". The driver may determine whether to cancel the order based on the voice reminder.

[0114] In some embodiments, when the photo violation confidence D is less than or equal to the second photo violation confidence threshold d2, or the driver has not cancelled the order, it can be determined that the driver has not violated the transportation regulations. The specific values ​​of the thresholds (d2.1, d2) can be set according to actual needs.

[0115] In some embodiments, when executing step S70, specifically, step S70 may further include the following steps:

[0116] Step S741: According to the illegal image information in the real-time video information and its corresponding video illegal confidence, the corresponding illegal behavior is determined:

[0117] Step S742: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal;

[0118] Step S743: When it is determined that the violation is that the customer wishes to transport in violation of regulations, the driver is fed back information about the order being cancelled without liability.

[0119] In some embodiments, when executing step S731 and step S732, specifically, the first video violation confidence threshold e1 can be used to determine the threshold of the video violation confidence E. The video violation confidence E is greater than e1, indicating that there is illegal passenger carrying content with a high violation confidence in the video information. At this time, the system will cancel the order immediately. The system sends a prompt message to the driver and the user, clearly informing them of the reason for the cancellation of the order, for example: "There is illegal passenger carrying behavior in the cargo box with a high violation confidence in your video, and the system automatically cancels this order. Please ensure that your cargo transportation complies with relevant laws and regulations."

[0120] In some embodiments, when step S733 is executed, specifically, when the video violation confidence E ≤ the first video violation confidence threshold e1 and e> the second video violation confidence threshold e2, it is determined that the client wants to transport in violation of regulations, and feedback is given to the driver to cancel the order without responsibility. The second video violation confidence threshold e2 can be divided into a second video violation confidence upper threshold e2.1 and a second video violation confidence threshold e2.

[0121] In some embodiments, when the video violation confidence level E ≤ the first video violation confidence threshold e1 and E> the second video violation confidence upper threshold e2.1, the specific processing may be a pop-up window reminder, and a pop-up window reminder is displayed on the driver side, "If the user wants the cargo box to carry passengers illegally, the order can be canceled without liability." The driver can click a button in the pop-up window to cancel the order directly.

[0122] In some embodiments, when the video violation confidence level E ≤ the second video violation confidence upper threshold value e2.1 and E> the second video violation confidence threshold value e2, the specific processing may be a voice broadcast reminder, in which the driver is reminded through voice broadcast that "the cargo box cannot be illegally loaded with people, and the order can be cancelled without liability". The driver can determine whether to cancel the order based on the voice reminder.

[0123] In some embodiments, when the video violation confidence E is less than or equal to the second video violation confidence threshold e2, or the driver has not cancelled the order, it can be determined that the driver has not violated the transportation regulations. The specific values ​​of the thresholds (e2.1, e2) can be set according to actual needs.

[0124] In some embodiments, when executing step S70, specifically, step S70 may further include the following steps:

[0125] Step S751: According to the illegal text information in the call text information and its corresponding call violation confidence, the illegal text information in the text chat information and its corresponding chat violation confidence, the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, and the illegal picture information in the real-time video information and its corresponding video violation confidence, the corresponding illegal behavior is determined:

[0126] Step S752: When it is determined that the violation is that the driver agrees to illegal transportation, the corresponding violation information is fed back to the driver and the client.

[0127] In some embodiments, when executing step S751, specifically, in the freight logistics monitoring system, in order to more accurately identify and handle the behavior of illegal loading of cargo boxes, the system not only sets a multi-level violation confidence threshold for a single information source, but also introduces the photo violation confidence D of the image recognition result as solid evidence, and combines it with the violation confidence of voice calls and text chats for judgment. This ensures that in voice and text messages with lower violation confidence, if the image recognition result provides further evidence support, the system will take more stringent measures.

[0128] In some embodiments, when step S751 is executed, specifically, condition X can be expressed as: call violation confidence B> third call violation confidence threshold b3, or chat violation confidence C> third chat violation confidence threshold c3. Condition Y can be expressed as: photo violation confidence D> third photo violation confidence threshold d3. If conditions X and Y are met at the same time, the order is directly canceled and the driver / user is prompted not to illegally carry people in the cargo box. The prompt message can be expressed as: sending a prompt message to the driver and the user, "There are medium and high violation confidence levels in the voice calls and photos in your order. The system automatically cancels this order. Please ensure that your cargo transportation complies with relevant laws and regulations." The interventions in the above stages all occur before the truck is officially transported, and intervention in illegal orders can be performed before the driver starts to execute the order. Among them, the specific size of the thresholds (b3, c3, d3) can be set according to actual needs.

[0129] In some embodiments, further, if the previous links have not identified illegal passenger transport with high violation confidence, and the combined channel identification result has not cancelled the order, the driver may still illegally transport passengers in the cargo box during the mid-trip stage of the truck. At this time, the order has not been cancelled, so it is necessary to integrate the previously obtained B, C, D, and E violation confidences. If the combination strategy is met, the corresponding order is cancelled. When any one of the call violation confidence B is greater than the fourth call violation confidence threshold b4, the chat violation confidence C is greater than the fourth chat violation confidence threshold c4, and the photo violation confidence D is greater than the fourth photo violation confidence threshold d4, and the video violation confidence E is greater than the third video violation confidence threshold e3, the order is cancelled and a voice prompt is given to the driver / user that the cargo box cannot be illegally transported. The intervention in the above stages occurs during truck transportation and belongs to mid-trip intervention. Among them, the specific size of the thresholds (b4, c4, d4, e3) can be set according to actual needs.

[0130] In some embodiments, if the violation confidence levels of B, C, D, and E do not meet the above conditions, it means that the order may be a normal order and the driver can complete the order normally.

[0131] In some embodiments, after steps S713, S723, S733, and S743, that is, after determining that the client wants to transport in violation of regulations and feeding back information about the order cancellation without liability to the driver, the processing method may further include the following steps:

[0132] Determine whether the driver cancels the order: if so, end the order; otherwise, review the order and punish the driver based on the review results.

[0133] In some embodiments, specifically, when the confidence level of the violation of the photo D>d2, or the confidence level of the violation of the video E>e2, and the order is directly cancelled, the order is sent for manual review: the loading diagram and the video clip of the corresponding order are sent for manual review. If the manual review confirms that the driver does have "illegal loading behavior in the cargo box", the driver will be punished accordingly. Specific measures include account suspension, point deduction, etc., depending on the number of violations the driver has committed during this period. The intervention at the above stage occurs after the order is completed and is a post-trip disposal.

[0134] In some embodiments, when the confidence level of the violation of the photo D>d2, or the confidence level of the violation of the video E>e2, and the driver has not cancelled the order, the order will be sent for manual review: the loading diagram and the video clip of the corresponding order will be sent for manual review. If the manual review confirms that the driver has indeed "violated the rules of loading passengers in the cargo box", the driver will be punished accordingly. Specific measures include account suspension, point deduction, etc., depending on the number of violations the driver has committed during this period. The intervention at the above stage occurs after the order is completed and is a post-trip disposal.

[0135] It can be seen that in the above scheme, by combining multiple information source recognition technologies and formulating different disposal strategies for different stages (before, during and after the trip), the platform can more accurately and timely identify and handle illegal cargo box loading behaviors. This comprehensive technical solution not only improves the accuracy of recognition and reduces the possibility of false alarms, but also effectively protects the personal safety of drivers and users, and enhances the supervision efficiency and user trust of the platform. By combining the recognition results of multiple information sources such as voice, text, images and videos, the recognition accuracy of illegal cargo loading behaviors is greatly improved. Multi-level violation confidence thresholds are set to ensure that reasonable judgments can be made under different violation confidence conditions. In the pre-trip stage, if a violation with a high violation confidence is identified, the system will immediately cancel the order to prevent the occurrence of violations. In the pre-trip and mid-trip stages, if a violation with a lower violation confidence is identified, the system will remind the driver through a pop-up window or voice to correct the potential violation in a timely manner. By setting multiple thresholds and introducing manual review links, the possibility of false alarms is effectively reduced, ensuring that the punishment of drivers is reasonable and fair. Cancel high-risk orders in the pre-trip stage to prevent violations at the source. During the journey, real-time video recording is used for monitoring to promptly detect and handle violations and ensure transportation safety. During the post-trip stage, violations are confirmed through manual review and drivers are penalized to eliminate the regulatory risks of the platform.

[0136] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0137] See also Figure 2 The present invention also provides a processing device for illegal transportation of freight vehicles, which can be applied to the above processing method. The processing device can include a preprocessing module 100, an information acquisition module 200, a semantic processing module 300, an image processing module 400, and a violation judgment module 500.

[0138] In some embodiments, the preprocessing module 100 can be used to obtain the client's order-based order note information; input the order note information into the semantic recognition model for semantic recognition to obtain the illegal text information in the order note information and its corresponding note violation confidence; judge the note violation confidence and the note violation confidence threshold: when the note violation confidence is greater than the note violation confidence threshold, it is determined that the client wishes to transport in violation of regulations, and the violation information is fed back to the driver; when the note violation confidence is less than or equal to the note violation confidence threshold, the client is allowed to place an order.

[0139] In some embodiments, the information acquisition module 200 can be used to obtain chat information between the driver and the client based on the order and image information of the freight vehicle during the transportation process.

[0140] In some embodiments, the semantic processing module 300 may be used to input chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information.

[0141] In some embodiments, the image processing module 400 may be used to input image information into an image recognition model to perform image recognition and obtain corresponding illegal picture information.

[0142] In some embodiments, the violation determination module 500 may be used to determine whether the order has any illegal transportation based on the illegal text information and the illegal picture information.

[0143] The embodiments of the present invention disclosed above are only used to help illustrate the present invention. The embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for handling illegal transportation by freight vehicles, characterized in that: include: Obtain chat information between the driver and the client based on the order, as well as image information of the freight vehicle during transportation; Inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information; Inputting the image information into an image recognition model for image recognition to obtain corresponding illegal picture information; Based on the illegal text information and / or illegal picture information, determine whether the order has any illegal transportation.

2. The method for handling illegal transportation of freight vehicles according to claim 1 is characterized in that: The chat information includes voice call information and text chat information; the image information includes cargo box photo information and real-time video information; the steps of inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information, and inputting the image information into an image recognition model for image recognition to obtain corresponding illegal picture information include: Convert the voice call information into text to generate corresponding call text information; Inputting the call text information and the text chat information into a semantic recognition model for semantic recognition respectively, and obtaining the illegal text information in the call text information and its corresponding call violation confidence, and the illegal text information in the text chat information and its corresponding chat violation confidence; The cargo box photographic information and the real-time video information are respectively input into an image recognition model for image recognition, so as to obtain the illegal picture information in the cargo box photographic information and its corresponding confidence level of the photographic violation, and the illegal picture information in the real-time video information and its corresponding confidence level of the video violation.

3. The method for handling illegal transportation of freight vehicles according to claim 2 is characterized in that: The step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes: According to the illegal text information in the call text information and its corresponding call violation confidence level, the corresponding illegal behavior is determined: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal; When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

4. The method for handling illegal transportation of freight vehicles according to claim 2 is characterized in that: The step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes: According to the illegal text information in the text chat information and its corresponding chat violation confidence, the corresponding illegal behavior is judged: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal; When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

5. The method for handling illegal transportation of freight vehicles according to claim 2, characterized in that: The step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes: According to the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, the corresponding illegal behavior is determined: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal; When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

6. The method for handling illegal transportation of freight vehicles according to claim 2, characterized in that: The step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes: According to the illegal image information in the real-time video information and its corresponding video violation confidence, the corresponding illegal behavior is judged: When it is determined that the violation is that the driver agrees to transport in violation of regulations, corresponding violation information is fed back to the driver terminal and the client terminal; When it is determined that the violation is that the customer wishes to transport in violation of regulations, the information of canceling the order without liability will be fed back to the driver.

7. The method for handling illegal transportation of freight vehicles according to claim 2, characterized in that: The step of determining whether the order has illegal transportation based on the illegal text information and / or illegal picture information includes: According to the illegal text information in the call text information and its corresponding call violation confidence, the illegal text information in the text chat information and its corresponding chat violation confidence, the illegal picture information in the cargo box photo information and its corresponding photo violation confidence, and the illegal picture information in the real-time video information and its corresponding video violation confidence, the corresponding illegal behavior is determined: When it is determined that the violation is that the driver agrees to illegal transportation, the corresponding violation information is fed back to the driver side and the client side.

8. The method for handling illegal transportation of freight vehicles according to any one of claims 3 to 6, characterized in that: After the step of feeding back information of canceling the order without liability to the driver side when judging that the illegal behavior is that the customer wants to transport in violation of regulations, the step further includes: Determine whether the driver cancels the order: If so, ending the order; Otherwise, the order will be reviewed and the driver will be punished based on the review results.

9. The method for handling illegal transportation of freight vehicles according to claim 1, characterized in that: Before the step of obtaining the chat information between the driver and the client based on the order and the image information of the freight vehicle during the transportation process, it also includes: Get the client's order-based order note information; Inputting the order note information into a semantic recognition model for semantic recognition, and obtaining the illegal text information in the order note information and its corresponding note violation confidence; According to the violation text information and its corresponding remark violation confidence, the corresponding violation behavior is determined: When it is determined that the illegal behavior is that the customer wishes to transport in violation of regulations, the corresponding illegal information is fed back to the client.

10. A device for handling illegal transportation of freight vehicles, characterized in that: include: The information acquisition module is used to obtain the chat information between the driver and the client based on the order and the image information of the freight vehicle during the transportation process; A semantic processing module, used for inputting the chat information into a semantic recognition model for semantic recognition to obtain corresponding illegal text information; An image processing module is used to input the image information into an image recognition model for image recognition to obtain corresponding illegal picture information; as well as The violation judgment module is used to determine whether there is any illegal transportation in the order based on the illegal text information and the illegal picture information.

Images