Cargo loading normativity evaluation method and device and storage medium

Abstract

The invention discloses a cargo loading normalization evaluation method and device and a storage medium, and relates to the technical field of logistics loading, and the method comprises the steps: obtaining a to-be-evaluated image; based on the to-be-evaluated image, carrying out segmentation identification to obtain a container area and a cargo area; detecting and evaluating the area image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box; determining a second evaluation result of the vacant space in the container based on the container area and the cargo area; and based on the first evaluation result and the second evaluation result, determining a normative evaluation result in the cargo loading process. According to the invention, the standardization of the cargo placement in the cargo box and the vacant space in the cargo box are evaluated, so that whether cargo loading is standard or not is scored after cargo loading of a truck is completed, drivers and customers can be reminded in time if the score of standardization evaluation is low, the customer complaint rate corresponding to a freight platform is reduced, and the cargo loading efficiency is improved. And the use experience of the customer is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics loading, and in particular to a method, device and storage medium for evaluating the standardization of cargo loading. Background Art

[0002] In the freight moving business, after the customer places an order, the freight driver uses a truck to transport the customer's moving items to the destination. In the process of loading the moving items onto the truck, the following problems usually occur. First, there may be unreasonable situations when the goods are loaded and placed, specifically: electrical appliances (such as refrigerators and washing machines, etc.) are placed upside down, which may cause damage to the appliances. Fragile items (such as mirrors, vases, and monitors, etc.) are not placed correctly and may be broken during transportation. Overly high objects (such as refrigerators or cabinets) are not fixed with ropes or supported by other suitable objects around them, which may cause the goods to fall over and be damaged during driving. Secondly, sometimes the customer's goods are small in quantity and only require small freight vehicles for transportation. However, drivers of large trucks may grab orders in order to make money, resulting in a large amount of vacant space in the large cargo box, which not only causes a waste of resources, but also may cause customers to pay higher fees for it.

[0003] These problems may lead to customer complaints and affect customers’ favorability towards the freight logistics platform. If these problems cannot be effectively resolved, customers may reduce their willingness to place orders on the platform. Summary of the invention

[0004] Based on this, it is necessary to provide a cargo loading standardization assessment method, device and storage medium for the above-mentioned technical problems, so as to solve at least one of the above-mentioned technical problems.

[0005] The present invention provides a method for evaluating the standardization of cargo loading, comprising:

[0006] Acquire an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded onto a truck;

[0007] Based on the image to be evaluated, segment and identify the cargo box area and the cargo area;

[0008] Performing detection and evaluation on the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box;

[0009] Determining a second evaluation result of vacant space in the cargo box based on the cargo box area and the cargo area;

[0010] Based on the first evaluation result and the second evaluation result, a normative evaluation result of the cargo loading process is determined.

[0011] Optionally, according to a cargo loading standardization evaluation method provided by the present invention, the detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement standard in the cargo box includes:

[0012] Performing target detection on the regional image to obtain a first detection result, wherein the first detection result includes a first object category and a first confidence level;

[0013] If the first object category is a first preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the first confidence level, wherein the first preset object category represents an object category that cannot be inverted during loading.

[0014] Optionally, according to a cargo loading standardization evaluation method provided by the present invention, the detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement standard in the cargo box includes:

[0015] Performing target detection on the regional image to obtain a second detection result, wherein the second detection result includes a first detection frame, a second object category, and a second confidence level;

[0016] If the second object category is a second preset object category, determining whether the aspect ratio corresponding to the first detection frame is greater than a preset aspect ratio threshold, wherein the second preset object category represents an object category that needs to be fixedly placed during loading;

[0017] If the aspect ratio is greater than a preset threshold, then based on the regional image, a third object category, location information, and a third confidence level are identified;

[0018] If the third object category is a third preset object category and the location information is within the preset area range of the first detection frame, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and the third confidence level, wherein the third preset object category represents an object category used for bundling objects.

[0019] Optionally, according to a method for evaluating the standardization of cargo loading provided by the present invention, if the height-to-width ratio is greater than a preset height-to-width ratio threshold, the method further includes:

[0020] Based on the regional image, segmenting to obtain an object region within a preset range from the first detection frame;

[0021] determining a relative position between an object in the object region and an object belonging to a second predetermined object category;

[0022] If the size height of the bounding box of the object area exceeds a preset height and the relative position is less than a preset distance threshold, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and a fourth confidence level associated with the object area.

[0023] Optionally, according to a cargo loading standardization evaluation method provided by the present invention, the detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement standard in the cargo box includes:

[0024] Detecting the regional image to obtain a third detection result, wherein the third detection result includes detection frames and confidence levels of different object categories;

[0025] If the object category in the third detection result includes a fourth preset object category, determining a target detection frame corresponding to an object belonging to the fourth preset object category, wherein the fourth preset object category represents an object category that is fragile during loading;

[0026] Among the object categories in the third detection result, determining whether there is a fifth preset object category within the preset distance of the target detection frame, wherein the fifth preset object category represents an object category with a hard material;

[0027] If there is a fifth preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the confidence associated with the fourth preset object category and the confidence associated with the fifth preset object category.

[0028] Optionally, according to a cargo loading standardization evaluation method provided by the present invention, the determining a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area includes:

[0029] Determining an area ratio based on the cargo box area and the cargo area;

[0030] Determining a target confidence based on the confidence associated with the cargo box region and the confidence associated with the cargo region;

[0031] If the area ratio is less than the preset area ratio threshold, and the target confidence is higher than the preset confidence threshold, a second evaluation result of the vacant space in the cargo box is determined based on the target confidence.

[0032] Optionally, according to a cargo loading normativeness assessment method provided by the present invention, determining a normativeness assessment result in a cargo loading process based on the first assessment result and the second assessment result includes:

[0033] The first evaluation result and the second evaluation result are weighted averaged to obtain the normative evaluation result.

[0034] Optionally, according to a cargo loading standardization assessment method provided by the present invention, the segmentation and identification of the cargo box area and the cargo area based on the image to be assessed includes:

[0035] Segmenting the image to be evaluated to obtain the cargo box area;

[0036] The regional image corresponding to the cargo box area is segmented to obtain the cargo area.

[0037] The present invention also provides a device for evaluating the standardization of cargo loading, comprising:

[0038] An acquisition module, used to acquire an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded onto a truck;

[0039] A segmentation module, used for segmenting and identifying a cargo box area and a cargo area based on the image to be evaluated;

[0040] A detection and evaluation module, used to detect and evaluate the regional image corresponding to the cargo box area, and obtain a first evaluation result of the cargo placement specification in the cargo box;

[0041] A first determination module, configured to determine a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area;

[0042] The second determination module is used to determine the normative evaluation result in the cargo loading process based on the first evaluation result and the second evaluation result.

[0043] The present invention also provides one or more readable storage media storing computer-readable instructions, which, when executed by a processor, implement the above-mentioned cargo loading standardization assessment method.

[0044] The above-mentioned method, device and storage medium for evaluating the standardization of cargo loading include: obtaining an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded; based on the image to be evaluated, segmenting and identifying the cargo box area and the cargo area; detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the standardization of the cargo placement in the cargo box; based on the cargo box area and the cargo area, determining a second evaluation result of the vacant space in the cargo box; based on the first evaluation result and the second evaluation result, determining the standardization evaluation result during the cargo loading process. The embodiment of the present invention evaluates the standardization of the placement of the cargo in the cargo box and the vacant space in the cargo box, thereby scoring whether the cargo loading is standardized after the truck is loaded. Drivers and customers with low standardization evaluation scores can be reminded in time, thereby reducing the corresponding customer call complaint rate of the freight platform and improving the customer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0046] Figure 1 is a flow chart of a method for evaluating the standardization of cargo loading in one embodiment of the present invention;

[0047] Figure 2 2 is a schematic diagram of a structure of a device for evaluating the standardization of cargo loading in one embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0051] In one embodiment, specifically, Figure 1 As shown, Figure 1 1 is a flow chart of a method for evaluating the standardization of cargo loading in an embodiment of the present invention. The embodiment of the present invention provides a method for evaluating the standardization of cargo loading, comprising the following steps:

[0052] Step S11, obtaining an image to be evaluated;

[0053] It should be noted that the image to be evaluated refers to the image of the cargo box after the goods are loaded. Optionally, the image to be evaluated refers to the image that needs to include the entire area of ​​the cargo box and the rear door of the carriage is fully open. In more detail: first, the image to be processed is obtained. The image to be processed can be any image, or the video can be decoded and framed to obtain several images, and any one of the images is selected. Optionally, the image acquisition can be directly captured by a camera, or a video or image can be obtained from a local or remote video library or image library. Then, it is determined whether the image is an image inside the cargo box, which completely includes the entire area of ​​the cargo box and the rear door of the carriage is in a fully open state. In one embodiment, a deep learning recognition model can be pre-trained to determine whether the image meets the requirements. For example, the image database prepared for training includes images inside the cargo box and the images include the entire cargo box area, the rear door of the carriage is in a fully open state, and the category label is "correct image"; and other scene images with the category label "wrong image", for example, the rear door of the carriage is not opened, not fully opened, the cargo box area is not photographed, the camera angle is too local, etc. The trained model can be used to evaluate whether the image meets the requirements, thereby obtaining the image to be evaluated.

[0054] Step S12, based on the image to be evaluated, segment and identify the cargo box area and the cargo area;

[0055] It should be noted that the cargo box area refers to the image area including the overall outline and boundaries of the cargo box, and the cargo area refers to the image area where the cargo is placed in the cargo box. The cargo area includes the outlines and boundaries of all cargo.

[0056] Specifically, the target detection algorithm or image segmentation algorithm can be used to segment and identify the image to be evaluated to obtain the cargo box area. For example, semantic segmentation models such as RefineNet and Unet or instance segmentation models such as HTC can be used to identify the cargo box area, or algorithms such as YOLO can be used to detect the cargo box area. In addition, the regional image corresponding to the cargo box area is segmented and identified to obtain the cargo area.

[0057] Step S13, detecting and evaluating the area image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box;

[0058] It should be noted that the assessment of the proper placement of goods in the cargo box includes scoring and assessment of scenarios such as whether objects such as refrigerators or washing machines are inverted, whether objects such as refrigerators or cabinets that are too high are fixed with ropes or supported by objects on all sides, and whether fragile items such as mirrors, flower pots, and monitors are surrounded by hard objects.

[0059] In one embodiment, for a scenario where objects such as refrigerators or washing machines are placed upside down, the specific evaluation process is as follows: target detection is performed on the regional image to obtain a first detection result, wherein the first detection result includes a first object category and a first confidence level. When the detected first object category contains an object category that cannot be inverted, a first score is calculated based on the first confidence level. The detailed evaluation process of the first score is specifically described in the following embodiments and will not be repeated here. Thus, the first score can be used as the first evaluation result of the cargo placement specification.

[0060] In one embodiment, for the scenario of whether objects such as refrigerators or cabinets that are too high are fixed with ropes, the specific evaluation process is as follows: target detection is performed on the regional image to obtain a second detection result, wherein the second detection result includes a first detection frame, a second object category, and a second confidence level; if the second object category is an object category that needs to be fixedly placed, then determine whether the aspect ratio corresponding to the first detection frame is greater than a preset aspect ratio threshold. Further, if it is greater than the preset aspect ratio threshold, it is determined that the object needs to be fixedly placed, and then use the recognition algorithm to determine whether there are objects such as ropes on the object that needs to be fixedly placed to fix it. Optionally, based on the regional image, a third object category, position information, and a third confidence level are identified. If the third object category is an object category used for bundling objects, and the position information is within the preset area range of the first detection frame, then it is determined that there are ropes around the object that needs to be fixedly placed to fix it, and then determine the second score based on the second confidence level and the third confidence level, wherein the detailed evaluation process of the second score is specifically described in the following embodiments and will not be repeated here. The second score can then be used as the first evaluation result of the cargo placement specification in the cargo box.

[0061] In one embodiment, for a scene where there are other objects around an object that needs to be fixedly placed, such as a refrigerator or cabinet that is too high, the specific evaluation process is as follows: based on the regional image, the object area within the preset range of the first detection frame is segmented, and the preset range can be set according to the actual situation, and then the boundary box corresponding to the object in the object area is determined. If the size height of the boundary box exceeds the preset height, and the relative position between the object in the object area and the object that needs to be fixed is less than the preset distance threshold, it is determined that there are other objects in the object area that can resist the object that needs to be fixed. The preset height can be set according to the actual situation, and then based on the second confidence and the fourth confidence associated with the object area, the third score is determined, wherein the detailed evaluation process of the third score is specifically described in the following embodiments and will not be repeated here. Therefore, the third score can be used as the first evaluation result of the cargo placement specification in the cargo box.

[0062] In other embodiments, the standardization of the placement of goods can be evaluated by combining the scenarios of whether the objects to be fixed are fixed with ropes and whether there are other objects around them to obtain a comprehensive score. For example, the second score and the third score are averaged to calculate the comprehensive score.

[0063] In one embodiment, for the scenario of whether fragile items are against hard objects, the specific evaluation process is as follows: the regional image is detected to obtain a third detection result, wherein the third detection result includes detection frames and confidences of different object categories, and then it is determined whether there is a fragile object category in the loading process in the third detection result. If there is a fragile object category, then in the detection frames of different object categories, the target detection frame of the fragile object category is determined, and then it is determined whether there is an object category of hard material within a preset distance of the target detection frame. The preset distance can be set according to actual conditions. If there is an object category of hard material within the preset distance, then based on the confidence associated with each fragile object category and the confidence associated with the object category of hard material, a fourth score is calculated, wherein the detailed evaluation process of the fourth score is specifically described in the following embodiments and will not be repeated here. Therefore, the fourth score can be used as the first evaluation result of the cargo placement specification.

[0064] In other embodiments, any two or more of the evaluation scenarios described above can be combined to comprehensively judge and evaluate the standardization of the placement of goods. For example, based on the first score of whether the evaluation object is placed upside down, the comprehensive score of whether the overly high object is fixed with ropes or supported by other objects on all sides, and the fourth score of whether the fragile items are against hard objects, a first evaluation result of the cargo placement standardization is formed, thereby achieving a comprehensive judgment and evaluation of the standardization of the cargo placement from multiple evaluation scenarios.

[0065] Step S14, determining a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area;

[0066] It should be noted that for the scenario where drivers of large trucks may grab orders in order to make money, resulting in a large vacant space in the large cargo box, the specific evaluation process is as follows: first, determine the size area of ​​the cargo box area and the size area of ​​the cargo area. Further, based on the size area of ​​the cargo box area and the size area of ​​the cargo area, determine whether there is a large vacant space in the cargo box, and then determine the second evaluation result of the vacant space in the cargo box according to the judgment result.

[0067] In more detail, in one embodiment, the area ratio is calculated based on the size area of ​​the cargo area and the size area of ​​the cargo box area, wherein the area ratio is calculated as follows: size area of ​​the cargo area / size area of ​​the cargo box area, and then based on the confidence associated with the cargo box area and the confidence associated with the cargo area, the target confidence is calculated. Furthermore, if the area ratio is less than a preset area ratio threshold and the target confidence is higher than a preset confidence threshold, it is determined that the vacant space is too large. The preset area ratio threshold and the preset confidence threshold are both set according to actual conditions, and then based on the target confidence, a second evaluation result of the vacant space in the cargo box is determined.

[0068] In another embodiment, the area of ​​the vacant space is obtained by subtracting the size area of ​​the cargo area from the size area of ​​the cargo box area, and then the area of ​​the vacant space is divided by the total area of ​​the cargo box area to obtain the proportion of the vacant space. Furthermore, if the calculated vacant space proportion is greater than a preset proportion threshold and the target confidence is higher than the preset confidence threshold, it is determined that the vacant space is too large. The preset proportion threshold is set according to the actual situation, and then based on the target confidence, a second evaluation result of the vacant space in the cargo box is determined.

[0069] It should be noted that the execution order of step S13 and step S14 is not limited to a certain order, and step S14 may be executed first and then step S13. In the case of parallel threads, step S13 and step S14 may also be executed simultaneously.

[0070] Step S15, determining a normative evaluation result during the cargo loading process based on the first evaluation result and the second evaluation result.

[0071] Specifically, the first evaluation result and the second evaluation result are weighted averaged, and the weighted average result is used as the normative evaluation result, so as to measure whether the entire cargo loading process is standardized in combination with the normative evaluation result.

[0072] In a specific embodiment, if the first evaluation result includes scores of multiple evaluation scenarios for different cargo placements, the scores of the evaluation scenarios for different cargo placements and the second evaluation results are weighted averaged to obtain a normative evaluation result. For example, the first score, the comprehensive score, the fourth score calculated in the above step S13 and the score in the second evaluation result in step S14 are weighted averaged to obtain a normative evaluation result.

[0073] In addition, in one embodiment of the present invention, after determining the normative evaluation result during the cargo loading process, if the normative evaluation result is lower than the preset evaluation score threshold, a prompt message will be generated and pushed to the driver and the passenger. The preset evaluation score threshold can be set according to the actual situation, so that the driver who receives the reminder can check the normativeness of the cargo placement again; at the same time, the customer who receives the reminder can know whether the current driver uses a large truck to take small orders, and whether there is any irregular loading.

[0074] The embodiment of the present invention, through the above scheme, includes: obtaining an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the goods are loaded; based on the image to be evaluated, segmenting and identifying to obtain a cargo box area and a cargo area; detecting and evaluating the regional image corresponding to the cargo box area, and obtaining a first evaluation result of the standard placement of the goods in the cargo box; based on the cargo box area and the cargo area, determining a second evaluation result of the vacant space in the cargo box; based on the first evaluation result and the second evaluation result, determining a standardization evaluation result during the cargo loading process. The embodiment of the present invention evaluates the standardization of the placement of the goods in the cargo box and the vacant space in the cargo box, thereby scoring whether the cargo loading is standardized after the truck is loaded. Drivers and customers with low standardization evaluation scores can be reminded in time, thereby reducing the corresponding customer call complaint rate of the freight platform and improving the customer experience.

[0075] In one embodiment of the present invention, the regional image corresponding to the cargo box area is detected and evaluated to obtain a first evaluation result of the cargo placement specification in the cargo box, including:

[0076] Step S21, performing target detection on the regional image to obtain a first detection result, wherein the first detection result includes a first object category and a first confidence level;

[0077] Step S22: If the first object category is the first preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the first confidence level.

[0078] It should be noted that the first preset object category represents an object category that cannot be inverted during loading, such as objects such as refrigerators and washing machines, wherein an inverted refrigerator or washing machine has different image texture features from a correctly placed refrigerator or washing machine. In this embodiment, target detection is performed on the regional image. For example, a target detection model can be used to detect whether there is an inverted refrigerator or washing machine in the cargo box area, and a first detection result is obtained by identification, wherein the first detection result includes a first object category and a first confidence corresponding to the first object category; if the first object category is the first preset object category, a first score is determined based on the first confidence, and then the first score is used as the first evaluation result of the cargo placement specification in the cargo box, wherein the larger the value of the first confidence, the higher the feasibility of the recognition result being an inverted refrigerator or washing machine. Optionally, the range of the first confidence is 0 to 1, and the calculation method of the first score is: 1-first confidence.

[0079] The embodiment of the present invention detects whether there are objects such as inverted refrigerators and washing machines during the loading process, thereby evaluating the standardization of the placement of goods in the cargo box and obtaining a first evaluation result of the standardization of the placement of goods.

[0080] In one embodiment of the present invention, the regional image corresponding to the cargo box area is detected and evaluated to obtain a first evaluation result of the cargo placement specification in the cargo box, including:

[0081] Step S31, performing target detection on the regional image to obtain a second detection result, wherein the second detection result includes a first detection frame, a second object category, and a second confidence level;

[0082] Step S32, if the second object category is a second preset object category, determining whether the aspect ratio corresponding to the first detection frame is greater than a preset aspect ratio threshold;

[0083] Step S33, if the aspect ratio is greater than the preset height-to-width ratio threshold, then based on the regional image, a third object category, location information and a third confidence level are identified;

[0084] Step S34: If the third object category is a third preset object category and the location information is within the preset area of ​​the first detection frame, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and the third confidence level.

[0085] It should be noted that the second preset object category represents the category of objects that need to be fixed during loading, such as cabinets, refrigerators and other objects that are too high; the third preset object category represents the category of objects used to bundle objects, such as ropes.

[0086] Specifically, target detection is performed on the regional image. For example, a target detection algorithm such as YOLO can be used to detect the second detection result. The second detection result includes a first detection frame, a second object category, and a second confidence level. The detection frame represents the position information of the object in the image. If the second object category is a second preset object category, the aspect ratio corresponding to the first detection frame is determined. The aspect ratio corresponding to the first detection frame is then compared with a preset aspect ratio threshold. The preset aspect ratio threshold can be set according to actual conditions. If it is greater than the preset aspect ratio threshold, it is determined that the height of the object is too high and is easy to fall over, so that it needs to be fixed with a rope.

[0087] Furthermore, a preset recognition algorithm can be used to determine whether the object to be fixed is fixed with a rope. Optionally, the rope around the outer periphery of the object to be fixed is considered to be fixed with a rope. Specifically, based on the regional image, a third object category, position information, and a third confidence level are identified. If the third object category is a third preset object category, and the position information of the object is within the preset area range of the first detection frame of the object to be fixed, it is determined that the object to be fixed is fixed with a rope, and then based on the second confidence level and the third confidence level, a second score is determined, and the second score is used as the first evaluation result of the cargo placement specification in the cargo box. Optionally, the second score is calculated as: second confidence level * third confidence level.

[0088] The embodiment of the present invention detects whether there are objects that are too high during the loading process, and then detects whether the identified objects that are too high are fixed with ropes, thereby evaluating the standardization of the placement of goods in the cargo box and obtaining an evaluation result of the standardization of the placement of goods.

[0089] In one embodiment of the present invention, if the aspect ratio is greater than a preset threshold, the method further includes:

[0090] Step S41, segmenting to obtain an object region within a preset range from a first detection frame based on the region image;

[0091] Step S42, determining the relative position between the object in the object area and the object belonging to the second preset object category;

[0092] Step S43, determining the relative position between the object in the object area and the object belonging to the second preset object category;

[0093] Step S44: If the size height of the bounding box of the object area exceeds the preset height and the relative position is less than the preset distance threshold, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and a fourth confidence level associated with the object area.

[0094] It should be noted that the standardization of the placement of goods can also be evaluated based on whether there are other objects around the object to be fixed to support and fix the object.

[0095] Specifically, the segmentation model is used to segment the regional image, so as to segment and identify the object region within the preset range of the first detection frame, and further determine the bounding box corresponding to the object in the object region, and determine the relative position between the object in the object region and the object belonging to the second preset object category (the object to be fixedly placed). If the size height of the bounding box exceeds the preset height, and the relative position is less than the preset distance threshold, it is determined that there is an object in the object region that can resist the object belonging to the second preset object category, wherein the preset height can be set according to the actual situation, for example, the preset height is set to half of the size height of the object to be fixedly placed. When it is monitored that both sides of the object belonging to the second preset object category are resisted by objects, it is determined that the object to be fixedly placed is resisted by objects around, and then based on the second confidence and the fourth confidence associated with the object region, the third score is determined, and then the third score is used as the first evaluation result of the cargo placement specification in the cargo box, wherein the confidence range of the fourth confidence associated with the object region is 0 to 1, and the larger the value, the higher the credibility of being resisted by the object. The third score is calculated as: second confidence * fourth confidence.

[0096] In other embodiments, the cargo placement specifications can be comprehensively evaluated by combining whether the over-high objects are tied up and whether the over-high objects are fixed by other surrounding objects, thereby determining a comprehensive score by combining the second confidence, the third confidence, and the fourth confidence. The comprehensive score is then used as the first evaluation result of the cargo placement specifications, and is calculated as follows: (P21*P22+P21*P23) / 2, wherein P21 represents the second confidence, P22 represents the third confidence, and P23 represents the fourth confidence, thereby improving the evaluation accuracy of the cargo placement specifications.

[0097] The embodiment of the present invention detects whether there are other objects around to support the objects that are too high, thereby evaluating the standardization of the placement of goods in the cargo box and obtaining an evaluation result of the standardization of the placement of goods.

[0098] In one embodiment of the present invention, the regional image corresponding to the cargo box area is detected and evaluated to obtain a first evaluation result of the cargo placement specification in the cargo box, including:

[0099] Step S51, detecting the regional image to obtain a third detection result, wherein the third detection result includes detection frames and confidence levels of different object categories;

[0100] Step S52, if the object category in the third detection result includes a fourth preset object category, determining a target detection frame corresponding to the object belonging to the fourth preset object category;

[0101] Step S53, among the object categories in the third detection result, determining whether there is a fifth preset object category within a preset distance of the target detection frame;

[0102] Step S54: if there is a fifth preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the confidence associated with the fourth preset object category and the confidence associated with the fifth preset object category.

[0103] It should be noted that the fourth preset object category represents the category of objects that are fragile during the loading process, for example, flower pots, mirrors, monitors, glass and other fragile objects; the fifth preset object category represents the category of objects with hard materials, for example, electrical appliances, cabinets, chairs and other objects with relatively hard materials.

[0104] Specifically, the regional image is detected using a preset instance segmentation model or a target detection model to obtain a third detection result, wherein the third detection result includes detection frames and confidences of different object categories. If the object category in the third detection result includes a fourth preset object category, it is determined that there is a fragile object, and then in each detection frame, the target detection frame corresponding to the object belonging to the fourth preset object category is determined. Further, among the object categories in the third detection result, it is determined whether there is a fifth preset object category within a preset distance of the target detection frame, that is, it is determined whether there are other harder objects around the fragile objects. If there is a fifth preset object category, a fourth score is calculated based on the confidence associated with the fourth preset object category (fragile object category) and the confidence associated with the fifth preset object category (hard object category), and the fourth score is used as the first evaluation result of the cargo placement specification in the cargo box. The fourth score is calculated as follows: 1-the confidence of each fragile item (i.e., the confidence associated with the fourth preset object category)*the confidence of the surrounding harder objects (i.e., the confidence associated with the fifth preset object category). The larger the value of the fourth score, the higher the standardization.

[0105] The embodiment of the present invention detects whether there are fragile objects and whether there are objects of hard materials around the fragile objects, thereby evaluating the standardization of the placement of goods in the cargo box and obtaining an evaluation result of the standardization of the placement of goods.

[0106] In one embodiment of the present invention, determining a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area includes:

[0107] Step S61, determining the area ratio based on the cargo box area and the cargo area;

[0108] Step S62, determining a target confidence based on the confidence associated with the cargo box area and the confidence associated with the cargo area;

[0109] Step S63: If the area ratio is less than the preset area ratio threshold and the target confidence is higher than the preset confidence threshold, a second evaluation result of the vacant space in the cargo box is determined based on the target confidence.

[0110] Specifically, the size area of ​​the cargo area is determined, and the size area of ​​the cargo box area is determined, and then the area ratio is calculated based on the size area of ​​the cargo area and the size area of ​​the cargo box area, wherein the area ratio is calculated as follows: size area of ​​the cargo area / size area of ​​the cargo box area. In addition, based on the confidence associated with the cargo box area and the confidence associated with the cargo area, the target confidence is calculated, wherein the target confidence is calculated as follows: (P41+P42) / 2, wherein P41 is the confidence of the cargo area, and P42 is the confidence of the cargo box area. If the area ratio is less than the preset area ratio threshold, and the target confidence is higher than the preset confidence threshold, then based on the target confidence, a fifth score is determined, and the fifth score is used as the second evaluation result of the vacant space in the cargo box. For example, when the area ratio is less than 1 / 2, and the target confidence is higher than the preset threshold, it means that there is vacant space in the cargo box, wherein the fifth score is calculated as follows: 1-target confidence. The larger the value of the fifth score, the higher the standardization of this item.

[0111] The embodiment of the present invention determines whether there is vacant space in the cargo box by detecting the area ratio between the cargo area and the cargo box area, and then determines the second evaluation result of the vacant space in the cargo box based on the confidence level associated with the cargo box area and the confidence level associated with the cargo area.

[0112] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0113] In one embodiment, a device for evaluating the standardization of cargo loading is provided, and the device for evaluating the standardization of cargo loading corresponds to the method for evaluating the standardization of cargo loading in the above embodiment. Figure 2 As shown, Figure 2 1 is a schematic diagram of a structure of a device for evaluating the standardization of cargo loading in one embodiment of the present invention, the device for evaluating the standardization of cargo loading comprises:

[0114] An acquisition module 21 is used to acquire an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded onto a truck;

[0115] A segmentation module 22, used for segmenting and identifying a cargo box area and a cargo area based on the image to be evaluated;

[0116] A detection and evaluation module 23 is used to detect and evaluate the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box;

[0117] A first determination module 24, configured to determine a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area;

[0118] The second determination module is used to determine the normative evaluation result in the cargo loading process based on the first evaluation result and the second evaluation result.

[0119] The detection and evaluation module 23 is also used for:

[0120] Performing target detection on the regional image to obtain a first detection result, wherein the first detection result includes a first object category and a first confidence level;

[0121] If the first object category is a first preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the first confidence level, wherein the first preset object category represents an object category that cannot be inverted during loading.

[0122] The detection and evaluation module 23 is also used for:

[0123] Performing target detection on the regional image to obtain a second detection result, wherein the second detection result includes the first detection frame, the second object category, and the second confidence level;

[0124] If the second object category is a second preset object category, determining whether the aspect ratio corresponding to the first detection frame is greater than a preset aspect ratio threshold, wherein the second preset object category represents an object category that needs to be fixedly placed during loading;

[0125] If the aspect ratio is greater than a preset threshold, then based on the regional image, a third object category, location information, and a third confidence level are identified;

[0126] If the third object category is a third preset object category and the location information is within the preset area of ​​the first detection frame, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and the third confidence level, wherein the third preset object category represents an object category used for bundling objects.

[0127] The detection and evaluation module 23 is also used for:

[0128] Based on the regional image, segmenting to obtain an object region within a preset range from the first detection frame;

[0129] determining a relative position between an object in the object region and an object belonging to a second predetermined object category;

[0130] If the size height of the bounding box of the object area exceeds the preset height and the relative position is less than the preset distance threshold, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and a fourth confidence level associated with the object area.

[0131] The detection and evaluation module 23 is also used for:

[0132] Detecting the regional image to obtain a third detection result, wherein the third detection result includes detection frames and confidence levels of different object categories;

[0133] If the object category in the third detection result includes a fourth preset object category, determining a target detection frame corresponding to the object belonging to the fourth preset object category, wherein the fourth preset object category represents a category of objects that are fragile during loading;

[0134] Among the object categories in the third detection result, determining whether there is a fifth preset object category within the preset distance of the target detection frame, wherein the fifth preset object category represents an object category with a hard material;

[0135] If there is a fifth preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the confidence associated with the fourth preset object category and the confidence associated with the fifth preset object category.

[0136] The first determining module 24 is further used for:

[0137] Determine the area ratio based on the cargo box area and cargo area;

[0138] Determining a target confidence based on the confidence associated with the cargo box region and the confidence associated with the cargo region;

[0139] If the area ratio is less than the preset area ratio threshold and the target confidence is higher than the preset confidence threshold, a second evaluation result of the vacant space in the cargo box is determined based on the target confidence.

[0140] The second determining module 25 is also used for:

[0141] Take a weighted average of the first and second evaluation results to obtain a normative evaluation result.

[0142] The segmentation module 22 is also used for:

[0143] Segment the image to be evaluated to obtain the cargo box area;

[0144] The regional image corresponding to the cargo box area is segmented to obtain the cargo area.

[0145] For the specific definition of the cargo loading normative evaluation device, please refer to the definition of the cargo loading normative evaluation method above, which will not be repeated here. Each module in the above cargo loading normative evaluation device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0146] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, Figure 3 : is a schematic diagram of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface and a database connected by a device bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating device, a computer-readable instruction and a database. The internal memory provides an environment for the operation of the operating device and the computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the method for evaluating the standardization of cargo loading. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a method for evaluating the standardization of cargo loading is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, and a network interface connected through a device bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a method for evaluating the standardization of cargo loading is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0148] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, and when the processor executes the computer-readable instructions, the steps of the above-mentioned method for evaluating the standardization of cargo loading are implemented. The method includes: obtaining an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded; based on the image to be evaluated, segmenting and identifying the cargo box area and the cargo area; detecting and evaluating the area image corresponding to the cargo box area, and obtaining a first evaluation result of the standardization of cargo placement in the cargo box; based on the cargo box area and the cargo area, determining a second evaluation result of the vacant space in the cargo box; based on the first evaluation result and the second evaluation result, determining the standardization evaluation result during the cargo loading process.

[0149] In one embodiment, a readable storage medium is provided, the readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by the processor to implement the steps of the above-mentioned cargo loading normative assessment method. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0150] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0151] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the standardization of cargo loading, characterized in that: include: Acquire an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded onto a truck; Based on the image to be evaluated, segment and identify the cargo box area and the cargo area; Performing detection and evaluation on the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box; Determining a second evaluation result of vacant space in the cargo box based on the cargo box area and the cargo area; Based on the first evaluation result and the second evaluation result, a normative evaluation result of the cargo loading process is determined.

2. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box includes: Performing target detection on the regional image to obtain a first detection result, wherein the first detection result includes a first object category and a first confidence level; If the first object category is a first preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the first confidence level, wherein the first preset object category represents an object category that cannot be inverted during loading.

3. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box includes: Performing target detection on the regional image to obtain a second detection result, wherein the second detection result includes a first detection frame, a second object category, and a second confidence level; If the second object category is a second preset object category, determining whether the aspect ratio corresponding to the first detection frame is greater than a preset aspect ratio threshold, wherein the second preset object category represents an object category that needs to be fixedly placed during loading; If the aspect ratio is greater than a preset threshold, identifying and obtaining a third object category, location information, and a third confidence level based on the regional image; If the third object category is a third preset object category and the location information is within the preset area range of the first detection frame, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and the third confidence level, wherein the third preset object category represents an object category used for bundling objects.

4. The method for evaluating the standardization of cargo loading according to claim 3 is characterized in that: If the aspect ratio is greater than a preset threshold, the method further includes: Based on the regional image, segmenting to obtain an object region within a preset range from the first detection frame; determining a relative position between an object in the object region and an object belonging to a second predetermined object category; If the size height of the bounding box of the object area exceeds a preset height and the relative position is less than a preset distance threshold, a first evaluation result of the cargo placement specification in the cargo box is determined based on the second confidence level and a fourth confidence level associated with the object area.

5. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The detecting and evaluating the regional image corresponding to the cargo box area to obtain a first evaluation result of the cargo placement specification in the cargo box includes: Detecting the regional image to obtain a third detection result, wherein the third detection result includes detection frames and confidence levels of different object categories; If the object category in the third detection result includes a fourth preset object category, determining a target detection frame corresponding to an object belonging to the fourth preset object category, wherein the fourth preset object category represents an object category that is fragile during loading; Among the object categories in the third detection result, determining whether there is a fifth preset object category within the preset distance of the target detection frame, wherein the fifth preset object category represents an object category with a hard material; If there is a fifth preset object category, a first evaluation result of the cargo placement specification in the cargo box is determined based on the confidence associated with the fourth preset object category and the confidence associated with the fifth preset object category.

6. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The determining, based on the cargo box area and the cargo area, a second evaluation result of the vacant space in the cargo box includes: Determining an area ratio based on the cargo box area and the cargo area; Determining a target confidence based on the confidence associated with the cargo box region and the confidence associated with the cargo region; If the area ratio is less than the preset area ratio threshold, and the target confidence is higher than the preset confidence threshold, a second evaluation result of the vacant space in the cargo box is determined based on the target confidence.

7. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The determining of the normative evaluation result in the cargo loading process based on the first evaluation result and the second evaluation result includes: The first evaluation result and the second evaluation result are weighted averaged to obtain the normative evaluation result.

8. The method for evaluating the standardization of cargo loading according to claim 1, characterized in that: The segmenting and identifying the cargo box area and the cargo area based on the image to be evaluated includes: Segmenting the image to be evaluated to obtain the cargo box area; The regional image corresponding to the cargo box area is segmented to obtain the cargo area.

9. A cargo loading standardization assessment device, characterized in that: include: An acquisition module, used to acquire an image to be evaluated, wherein the image to be evaluated refers to an image of a cargo box after the cargo is loaded onto a truck; A segmentation module, used for segmenting and identifying a cargo box area and a cargo area based on the image to be evaluated; A detection and evaluation module, used to detect and evaluate the regional image corresponding to the cargo box area, and obtain a first evaluation result of the cargo placement specification in the cargo box; A first determination module, configured to determine a second evaluation result of the vacant space in the cargo box based on the cargo box area and the cargo area; The second determination module is used to determine the normative evaluation result in the cargo loading process based on the first evaluation result and the second evaluation result.

10. A readable storage medium having computer readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the cargo loading compliance assessment method according to any one of claims 1 to 8 is implemented.

Images