Data Processing Method and System for Semi-High Container Management System

Abstract

The present invention provides a data processing method and system for a semi-high container management system, constructs a management model corresponding to a semi-high container yard, and the management model includes monitoring points corresponding to a plurality of container stacks; obtains side images taken by a collection device at the monitoring points for the corresponding container stacks, performs a safety status analysis on the side images that meet the angle recognition conditions according to an angle analysis model to obtain angle analysis data; performs a safety status analysis on the side images that meet the detail recognition conditions according to a multi-level monitoring model to obtain detail analysis data; generates safety monitoring data corresponding to the container stacks according to the angle analysis data or the detail analysis data, and updates the management model according to the safety monitoring data to obtain a safety monitoring model.

Description

Technical Field

[0001] The present invention relates to data processing technologies, and in particular to a data processing method and system for a semi-high container management system. Background Art

[0002] A semi-high container is a special container with a height of half the height of a standard container, mainly used for transporting heavy, large, bulk, or over-sized goods. To protect the safety of the goods, a door lock is generally configured for the semi-high container to prevent the goods from being lost, damaged, or stolen.

[0003] In the prior art, inspection personnel are generally dispatched to monitor the door lock status of each semi-high container. However, through this method, it may not be possible to give an early warning in a timely manner when the door lock status is abnormal, reducing the safety of the goods inside the container.

[0004] Therefore, how to monitor the door lock status of the container and improve the safety of the goods has become an urgent problem to be solved today. Summary of the Invention

[0005] The present invention provides a data processing method and system for a semi-high container management system, which can monitor the door lock status of the container and improve the safety of the goods.

[0006] In a first aspect of the present invention, there is provided a data processing method for a semi-high container management system, including:

[0007] Constructing a management model corresponding to a semi-high container yard, where the management model includes monitoring points corresponding to multiple container stacks;

[0008] Obtaining a side image captured by a collection device at the monitoring point for the corresponding container stack, and performing a safety status analysis on the side image that meets the angle recognition condition according to an angle analysis model to obtain angle analysis data;

[0009] Performing a safety status analysis on the side image that meets the detail recognition condition according to a multi-level monitoring model to obtain detail analysis data;

[0010] Generating safety monitoring data corresponding to the container stack according to the angle analysis data or the detail analysis data, and updating the management model according to the safety monitoring data to obtain a safety monitoring model.

[0011] Optionally, in a possible implementation manner of the first aspect, obtaining a side image captured by a collection device at the monitoring point for the corresponding container stack, and performing a safety status analysis on the side image that meets the angle recognition condition according to an angle analysis model to obtain angle analysis data includes:

[0012] Obtain the real-time shooting interface of the acquisition device at the monitoring point based on a preset downward shooting height, and extract the stack downward shooting contour in the real-time shooting interface. The center point of the stack downward shooting contour corresponds to the center point of the real-time shooting interface;

[0013] Determine the contour line corresponding to the end face side of the container stack in the stack downward shooting contour as the reference line. The end face side includes the front end with a door and the rear end without a door;

[0014] Connect the center points of the two reference lines to obtain a guiding line, and extend both ends of the guiding line respectively according to the overall shooting distance;

[0015] Determine the direction from the center point of the guiding line to the end point of the guiding line as the guiding direction, and control the acquisition device to move to the guiding point corresponding to the corresponding end point based on the guiding direction and the guiding line;

[0016] Obtain the end face side corresponding to the guiding point as the target side, and control the acquisition device to take images of the target side in the height direction at a side shooting interval distance to obtain a plurality of side images;

[0017] Perform a door lock structure analysis on the side images, determine the side images that meet the angle recognition conditions as the first target images, and perform a safety status analysis on the first target images according to the angle analysis model to obtain angle analysis data.

[0018] Optionally, in a possible implementation manner of the first aspect, obtaining the end face side corresponding to the guiding point as the target side, and controlling the acquisition device to take images of the target side in the height direction at a side shooting interval distance to obtain a plurality of side images, includes:

[0019] After the acquisition device reaches the guiding point, taking the guiding point as the reference point, control the acquisition device to move down to the side shooting position point in the height direction at the side shooting interval distance;

[0020] Obtain the side image taken by the acquisition device at the side shooting position point, extract the end face contour in the side image, and obtain 4 edge points corresponding to the first end face contour in the acquisition direction;

[0021] Generate an identification frame according to the edge points, and determine the image area located within the identification frame in the side image as the identification area;

[0022] Perform door lock recognition on the identification area. When there is a door lock structure in the identification area, determine that the side image meets the monitoring and recognition conditions;

[0023] When the door lock structure does not exist in the recognition area, it is determined that the side image does not meet the monitoring and recognition conditions, and the side image is deleted. The acquisition sequence number corresponding to the side image is determined as the target sequence number;

[0024] Obtain the current shooting height corresponding to the target sequence number, and determine the current shooting height as the target shooting height when the acquisition device performs side image acquisition on the other end face side;

[0025] Based on the side shooting position point, continue to control the acquisition device to move down to the next side shooting position point in the height direction according to the side shooting interval distance;

[0026] Repeat the above steps of identifying the door lock structure according to the side image until the acquisition sequence number reaches the preset acquisition sequence number corresponding to the container stack, and stop the acquisition of the side image of the target side.

[0027] Optionally, in a possible implementation manner of the first aspect, perform door lock structure analysis on the side image, determine the side image that meets the angle recognition condition as the first target image, and perform safety state analysis on the first target image according to the angle analysis model to obtain angle analysis data, including:

[0028] Obtain the side image that meets the monitoring and recognition conditions as the recognition image, and perform hanging lock recognition on the door lock handle combination corresponding to the door lock structure in the recognition image;

[0029] Determine that the side image of the door lock handle combination without a hanging lock meets the angle recognition condition, and obtain the side image as the first target image;

[0030] According to the angle analysis model, obtain the closing angle corresponding to each door lock handle combination in the first target image, and perform safety state analysis on the closing angle to obtain angle analysis data.

[0031] Optionally, in a possible implementation manner of the first aspect, according to the angle analysis model, obtain the closing angle corresponding to each door lock handle combination in the first target image, and perform safety state analysis on the closing angle to obtain angle analysis data, including:

[0032] Obtain the door handrest contour, door handle contour in each door lock handle combination in the first target image, and the door lock rod contour connected to each door lock handle combination;

[0033] Determine the connection point between the door lock rod contour and the door handrest contour, connect the center point of the door handle contour and the connection point to obtain a handle angle line, and connect the center point of the door handrest contour and the connection point to obtain a reference angle line;

[0034] Determine the closing angle between the handle inclination line and the reference angle line. When the closing angle is greater than or equal to the closing angle threshold, determine the corresponding door lock handle combination as an abnormal door lock combination;

[0035] When the closing angle is less than the closing angle threshold, determine the corresponding door lock handle combination as a normal door lock combination, and obtain angle analysis data based on the abnormal door lock combination and / or the normal door lock combination.

[0036] Optionally, in a possible implementation manner of the first aspect, perform a security status analysis on the side image that meets the detail recognition conditions according to a multi-level monitoring model to obtain detail analysis data, including:

[0037] Perform a door lock structure analysis on the side image, determine the side image that meets the detail recognition conditions as the second target image, and determine the corresponding detail acquisition set of the second target image according to the multi-level monitoring model;

[0038] Generate an outer contour frame of the detail acquisition set, obtain the contour center point of the outer contour frame, and determine the direction from the center point of the second target image to the contour center point as the adjustment direction;

[0039] Determine the difference distance between the center point of the second target image and the contour center point, and control the acquisition device to move to the first position point based on the adjustment direction and the difference distance;

[0040] Control the acquisition device to advance forward in the detail acquisition direction until the interface ratio of the overall combined contour corresponding to the detail acquisition set in the real-time shooting interface is within the optimal shooting ratio range, and then stop the movement of the acquisition device and determine the corresponding position point as the detail acquisition point;

[0041] Obtain the detail image captured by the acquisition device at the detail acquisition point, and perform a security identification on the hanging lock in the detail image to obtain detail analysis data.

[0042] Optionally, in a possible implementation manner of the first aspect, perform a door lock structure analysis on the side image, determine the side image that meets the detail recognition conditions as the second target image, and determine the corresponding detail acquisition set of the second target image, including:

[0043] Obtain the side image that meets the monitoring recognition conditions as the recognition image, and perform a hanging lock recognition on the door lock handle combination corresponding to the door lock structure in the recognition image;

[0044] Determine that the side image of the door lock handle combination with the wall-mounted lock meets the detail recognition condition, and obtain the side image as the second target image;

[0045] According to the multi-level monitoring model, obtain the combination contours corresponding to each door lock handle combination in the second target image, number each of the combination contours, and obtain the position numbers corresponding to each of the combination contours;

[0046] Based on the center points of each of the combination contours, determine the combination distances between each of the combination contours, and determine that the door lock handle combinations with the combination distances less than the door lock distance threshold are the same set of the detail collection sets, and add corresponding collection numbers to the detail collection sets according to the position numbers.

[0047] Optionally, in a possible implementation manner of the first aspect, obtain the detail images taken by the acquisition device at the detail acquisition points, perform security recognition on the wall-mounted locks in the detail images, and obtain detail analysis data, including:

[0048] Extract the wall-mounted lock contours corresponding to the wall-mounted locks in each of the door lock handle combinations in the detail images. When the wall-mounted lock contour is in the open state, determine the corresponding door lock handle combination as an abnormal door lock combination;

[0049] When the wall-mounted lock contour is in the closed state, retrieve the standard wall-mounted lock contour, and obtain a comparison value by comparing the wall-mounted lock contour with the standard wall-mounted lock contour. The standard wall-mounted lock contour includes a front standard contour and a back standard contour;

[0050] Obtain the wall-mounted lock contours with the comparison values greater than the similarity threshold as the secondary determination contours, and determine that the orientation of the secondary determination contour compared with the front standard contour is the positive orientation, and the orientation of the secondary determination contour compared with the back standard contour is the negative orientation;

[0051] Retrieve a plurality of reference container door images corresponding to the container stack. Each of the reference container door images is provided with a corresponding preset end face side and a preset acquisition sequence number;

[0052] According to the end face side and the acquisition sequence number of the side image corresponding to the detail image, determine the reference container door image corresponding to the detail image as the comparison container door image;

[0053] Determine the reference orientation of the reference wall-mounted lock contour corresponding to the position number of the secondary determination contour in the comparison container door image. If the orientation of the secondary determination contour is consistent with the reference orientation, determine the corresponding door lock handle combination as a normal door lock combination;

[0054] If the orientation of the secondary determination profile is inconsistent with the reference orientation, determine the corresponding door lock handle combination as an abnormal door lock combination, and obtain detailed analysis data based on the abnormal door lock combination and / or normal door lock combinations.

[0055] Optionally, in a possible implementation manner of the first aspect, generate safety monitoring data corresponding to the container stack based on the angle analysis data or the detailed analysis data, and update the management model according to the safety monitoring data to obtain a safety monitoring model, including:

[0056] Obtain a plurality of sub-containers corresponding to the container stack, and construct monitoring nodes corresponding to each of the sub-containers;

[0057] Arrange the monitoring nodes in descending order according to the stacking layers corresponding to each of the sub-containers, and connect them to obtain a monitoring structure;

[0058] Obtain the acquisition sequence numbers corresponding to each of the angle analysis data or the detailed analysis data, and bind the monitoring nodes and the angle analysis data or the detailed analysis data with the same acquisition sequence number and arrangement order;

[0059] Determine the monitoring nodes with the abnormal door lock combination as abnormal nodes, and determine the warning level of the abnormal nodes according to the abnormal quantity corresponding to the abnormal door lock combination;

[0060] Retrieve the pixel values corresponding to the corresponding warning level to update the abnormal nodes, obtain the safety monitoring data corresponding to the container stack, and bind the safety monitoring data with the corresponding monitoring points in the management model to obtain a safety monitoring model.

[0061] In a second aspect of the present invention, there is provided a data processing system for a semi-high container management system, including:

[0062] A management module for constructing a management model corresponding to a semi-high container yard, where the management model includes monitoring points corresponding to a plurality of container stacks;

[0063] An angle module for obtaining a side image captured by an acquisition device at the monitoring point for the corresponding container stack, and performing a safety status analysis on the side image that meets the angle recognition condition according to an angle analysis model to obtain angle analysis data;

[0064] A detail module for performing a safety status analysis on the side image that meets the detail recognition condition according to a multi-level monitoring model to obtain detailed analysis data;

[0065] A monitoring module, configured to generate safety monitoring data corresponding to the container stack according to the angle analysis data or the detail analysis data, and update the management model according to the safety monitoring data to obtain a safety monitoring model.

[0066] The beneficial effects of the present invention are as follows:

[0067] The present invention can monitor the door lock status of containers, improving the safety of goods. The present invention will dispatch collection devices to monitor the door lock status of each container in multiple container stacks in a semi-high container yard, so that when the door lock status of a container is abnormal, it can timely remind the management personnel, reducing the safety hazards of goods.

[0068] When the present invention monitors the door lock status of multiple container stacks in a semi-high container yard, it will perform status detection on them in different ways according to the type of container door lock. For door locks without hanging locks, this solution will judge whether it is in the closed state by the closing angle between the door handle and the door handle support, so that when the closing angle between the door handle and the door handle support is in an abnormal state, such as the open state, it can timely remind the management personnel. For door locks with hanging locks, this solution will control the drone to perform secondary magnified image acquisition on the door locks with hanging locks, making the door lock handle combination in the obtained image clearer, improving the accuracy during recognition, and during abnormal analysis, this solution will perform multi-level detection on the closing state and orientation of the hanging locks, thereby further improving the accuracy during recognition.

[0069] The present invention will also determine the corresponding warning level in combination with the number of abnormal door locks of each container, so as to warn containers with a lower safety factor. For different warning levels, the present invention will also use different pixel values to perform warning display for the management personnel, so that the management personnel can observe the safety level of the corresponding container through the pixel values and timely make targeted treatment strategies for containers with a lower safety factor. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a schematic flowchart of a data processing method for a semi-high container management system provided by an embodiment of the present invention;

[0071] Figure 2 is a schematic diagram of a collection device taking a side image provided by an embodiment of the present invention;

[0072] Figure 3 is a schematic diagram of a side image provided by an embodiment of the present invention;

[0073] Figure 4 is a schematic structural diagram of a data processing system for a semi-high container management system provided by an embodiment of the present invention. Detailed implementation manners

[0074] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0075] Refer to Figure 1 , which is a schematic flowchart of a data processing method for a semi-high container management system provided by an embodiment of the present invention. Figure 1 The execution subject of the method shown can be a software and / or hardware device. The execution subject of the present application may include, but is not limited to, at least one of the following: user equipment, network equipment, etc. Among them, the user equipment may include, but is not limited to, a computer, a smart phone, a personal digital assistant (Personal Digital Assistant, abbreviated as: PDA), and the electronic equipment mentioned above. The network equipment may include, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which is composed of a group of loosely coupled computers to form a super virtual computer. This embodiment does not make any restrictions on this. It includes steps S1 to S4, which are specifically as follows:

[0076] S1. Construct a management model corresponding to the semi-high container yard, where the management model includes monitoring points corresponding to multiple container stacks.

[0077] In practical applications, in order to ensure the safety of the goods in the semi-high container, a door lock is generally configured for the container. If the door lock is damaged, it may affect the safety of the goods in the container. Based on this, this solution will dispatch a collection device to monitor the door lock status of each container in multiple container stacks in the semi-high container yard, so that the management personnel can be reminded in time when the door lock status of the container is abnormal, reducing the safety hazard of the goods.

[0078] Among them, a semi-high container yard refers to a site for stacking semi-high containers. A container stack refers to a stack formed by multiple stacked semi-high containers. A management model refers to a twin model corresponding to the semi-high container yard. Managers can construct the management model accordingly based on the actual situation. This model can include monitoring points corresponding to multiple container stacks. Subsequently, acquisition devices can be dispatched to the corresponding monitoring points to collect images of the door locks of the container stacks and detect the door lock status of the collected image data, so as to be able to prompt the managers in a timely manner when the door lock status is abnormal. A monitoring point refers to a point for monitoring the door lock status corresponding to a container stack.

[0079] S2. Obtain the side image taken by the acquisition device of the corresponding container stack at the monitoring point, and perform a safety status analysis on the side image that meets the angle recognition condition according to the angle analysis model to obtain angle analysis data.

[0080] The above acquisition device can be a drone. Specifically, the drone can go to the monitoring point corresponding to the corresponding container stack to collect images of the end face side with a door on the side of the container stack, and then analyze the door lock status of the collected images, so as to be able to timely discover the containers with abnormal door lock status and remind the managers.

[0081] The side image refers to the image data taken by the drone of the side of the container stack. It can be understood that in this solution, when analyzing the door lock status, different methods of status analysis will be performed on the door locks with hanging locks and the door locks without hanging locks. For the door locks without hanging locks, this solution will judge whether it is in the closed state through the closing angle between the door handle and the door handle support, that is, use the angle analysis model to perform a safety status analysis on the door lock status. The method for analyzing the status of the door locks with hanging locks will be elaborated in detail below and will not be elaborated here.

[0082] Among them, the angle analysis model refers to a model for analyzing the status of the door lock components, such as the closing angle between the door handle and the door handle support. When the door lock components in the side image have hanging locks, it can be determined that the side image meets the angle recognition condition, that is, it meets the condition for analyzing the closing angle between the door lock components. The angle analysis data refers to the data obtained after analyzing the closed state of the door lock.

[0083] Based on the above embodiments, the specific implementation manner of step S2 can be:

[0084] S21. Obtain the real-time shooting interface of the acquisition device at the monitoring point based on the preset downward shooting height, and extract the stack downward shooting contour in the real-time shooting interface. The center point of the stack downward shooting contour corresponds to the center point of the real-time shooting interface.

[0085] See Figure 2 , which is a schematic diagram of a collection device for taking side images provided by an embodiment of the present invention. It can be understood that since a container stack is formed by stacking multiple containers, when the unmanned aerial vehicle (UAV) performs data collection, it can first fly above the container stack to determine the end face side for image collection, and then move to the corresponding end face side to collect images of the container doors on the side of the container. In practical applications, the height of the UAV when taking an overhead shot of the corresponding container stack can be pre-configured, and at this height, the UAV can capture the overall outline of the topmost container.

[0086] It can be understood that since the container doors are generally on the side, when the UAV takes an overhead shot of the container stack, it can determine the side where the end face side of the container door is located, and then move to the front of the corresponding side and then move down to take pictures of the sides of each layer of containers. For example, the UAV can move according to the Figure 2 shown movement trajectory, so as to take pictures of the sides of the corresponding level of containers.

[0087] Among them, the preset overhead shooting height refers to the height set in advance when the UAV takes an overhead shot, and the stack overhead shooting outline refers to the top outline corresponding to the topmost container when the UAV takes an overhead shot. It can be understood that in order to make the shooting position point of the UAV be located at the center position of the container stack, the center point of the stack overhead shooting outline can be aligned with the center point of the real-time shooting interface.

[0088] S22. Determine the contour line corresponding to the end face side of the container stack in the stack overhead shooting outline as the reference line, where the end face side includes a front end with a container door and a rear end without a container door.

[0089] It can be understood that the side of the container includes a front end face with a container door, a rear end face without a container door, and side wall faces on both sides. From a top view angle, its outline is generally rectangular. The lengths of the contour lines of the front end face and the rear end face are generally the same, and the lengths of the contour lines of the side wall faces on both sides are generally the same. Moreover, the contour lines corresponding to the front end and the rear end are generally shorter than the contour lines corresponding to the side wall faces. Therefore, the line segment with a shorter length in the stack overhead shooting outline can be determined as the contour line corresponding to the end face side of the container stack.

[0090] S23. Connect the center points of the two reference lines to obtain a guiding line, and extend both ends of the guiding line according to the overall shooting distance.

[0091] It can be understood that in order to enable the UAV to move to the front of the corresponding end face side and then move down to collect images of the corresponding end face side, a guiding line can be generated to guide the movement of the UAV, so that the UAV can move horizontally through the guiding line.

[0092] Among them, the guiding line is a line segment for guiding the horizontal movement of the drone. In order to enable the drone to move to the front of the end face side, the two ends of the guiding line can be extended so that the drone can move to the front position of the container stack. The overall shooting distance can be the shooting distance between the drone and the container when the drone takes a side view of the container. This shooting distance can be set in advance, and at this distance, the drone can capture an image corresponding to the entire side of the container.

[0093] S24. Determine the direction from the center point of the guiding line to the end point of the guiding line as the guiding direction, and control the acquisition device to move to the guiding point corresponding to the corresponding end point based on the guiding direction and the guiding line.

[0094] It can be understood that since it is impossible to determine which contour line corresponds to the front end of the container door and which contour line corresponds to the rear end without a container door from the top-down view, and when analyzing the door lock state, the door lock is generally configured on the end face where the container door is located. Therefore, it is necessary to analyze the captured image corresponding to the side of the container door. So, in this solution, the drone will first be controlled to fly to one side for shooting, and then analyze the door lock state of the side image corresponding to the container with a container door on this side, and record the containers without a container door. Thus, when shooting on the other side, the door image corresponding to the recorded container can be quickly captured.

[0095] Therefore, the above guiding direction includes two directions, that is, the directions pointing from the center point to the two ends of the guiding line respectively. When the drone is collecting data, it can first fly to one side for image collection through the guiding line and the guiding direction, then fly back to the original position, and then fly to the other side for image collection through the guiding line and the other guiding direction.

[0096] Among them, the guiding point refers to the position point where the drone is located after moving horizontally according to the guiding line and the guiding direction.

[0097] S25. Obtain the end face side corresponding to the guiding point as the target side, and control the acquisition device to take images of the target side in the height direction at intervals of the side shooting distance to obtain a plurality of side images.

[0098] The target side is the side where the drone is currently collecting images. When the drone moves down, it can move at intervals of the side shooting distance. The side shooting distance is the interval downward movement distance when the drone takes pictures of the container in the height direction. This interval distance can be set accordingly according to the length of the container in the height direction, so that the drone can capture images of each container in the height direction during shooting.

[0099] It is worth mentioning that when the drone collects images of the side of the topmost container, if the complete side image of the container cannot be captured after reducing the side shooting interval distance, the drone can continue to be controlled to move downward until the complete side image of the first container can be fully captured in the real-time shooting interface of the drone, and then the movement of the drone stops. When the drone takes subsequent shots, it can continue to move downward according to the side shooting interval distance for corresponding shooting.

[0100] In some embodiments, step S25 can be implemented through steps S251 to S258, specifically as follows:

[0101] S251, after the acquisition device reaches the guiding point, taking the guiding point as the reference point, controlling the acquisition device to move downward in the height direction to the side shooting position point according to the side shooting interval distance.

[0102] Among them, the side shooting position point refers to the position point where the drone shoots the side of the container at the corresponding level. For example Figure 2 as shown in, there are 4 layers of containers in total, then the above-mentioned side shooting position point can be the position point when the drone shoots the side of the topmost layer, that is, the 4th layer of containers.

[0103] S252, obtaining the side image captured by the acquisition device at the side shooting position point, extracting the end face contour in the side image, and obtaining 4 edge points corresponding to the first end face contour in the acquisition direction.

[0104] As Figure 2 shown, since the adjacent layers of containers are closely adjacent to each other, when shooting the upper layer of containers, a part of the upper layer of the lower layer of containers may also be captured in the image. Therefore, in order to exclude other interferences in the image, this solution will extract the end face part of the currently captured container for subsequent corresponding analysis.

[0105] See Figure 3 , which is a schematic diagram of a side image provided by an embodiment of the present invention. It can be understood that the end face contour of the container generally has corresponding edge points. Therefore, in order to determine the end face part of the container, the end face contour in the side image can be extracted, that is, the overall contour corresponding to the front end or the rear end of the container, and then the edge points located at the four corners of the end face contour can be determined, so as to be able to determine the corresponding end face area of the container through the edge points in the subsequent process. The edge points can be the 4 vertices of the end face contour.

[0106] S253, generating an identification frame according to the edge points, and determining the image area located within the identification frame in the side image as the identification area.

[0107] When generating the recognition frame, the four edge points can be connected to obtain a rectangular recognition frame, and then the image area within the recognition frame in the side image is used as the recognition area for subsequent analysis and recognition.

[0108] S254. Perform door lock recognition on the recognition area. When there is a door lock structure in the recognition area, determine that the side image meets the monitoring and recognition conditions.

[0109] It can be understood that since the collected side image may be the image corresponding to the front end face with a box door or the image corresponding to the back end face without a box door, and generally there is no door lock structure in the image corresponding to the back end face. Therefore, door lock recognition can be performed on the recognition area. If there is a door lock structure in the recognition area, it may correspond to the front end face and the door lock state can be monitored. Therefore, it can be determined that the corresponding side image meets the monitoring and recognition conditions, that is, meets the conditions for door lock state monitoring. Subsequently, abnormal analysis of the door lock state of the corresponding container can be performed based on this image. The door lock structure is the Figure 3 door lock handle combination shown in

[0110] S255. When there is no such door lock structure in the recognition area, determine that the side image does not meet the monitoring and recognition conditions, delete the side image, and determine the acquisition ordinal number corresponding to the side image as the target ordinal number.

[0111] When there is no such door lock structure in the recognition area, it indicates that the collected side image may be the image corresponding to the back end face without a box door, and its front end face may be on the other side. Therefore, it can be determined that the side image does not meet the monitoring and recognition conditions, and the side image is deleted.

[0112] Among them, the acquisition ordinal number refers to the number of times of side acquisition by the unmanned aerial vehicle. The acquisition ordinal number is associated with the number of layers of the container collected by the unmanned aerial vehicle. For example Figure 2 as shown in

[0113] When the unmanned aerial vehicle performs side acquisition on the 4th layer of the container, the corresponding target ordinal number is 1, that is, the 4th layer of the container is the container that is collected for the 1st time in the height direction. Since the front end and the back end correspond to each other, using this acquisition ordinal number as the target ordinal number can quickly determine the position point when the unmanned aerial vehicle performs image acquisition on the other side, so as to quickly collect the end face with a box door on the other side.

[0114] Specifically, after obtaining the target ordinal number, the acquisition height corresponding to the target ordinal number, i.e., the above-mentioned current shooting height, can be recorded, and this height can be bound to the other end face side. That is, it can be controlled that when the drone conducts data acquisition on the other side, it can quickly move down to the corresponding position point to conduct data acquisition on the door end face of the corresponding container, instead of sequentially acquiring the side images of the containers at the corresponding levels and then determining whether they are the images corresponding to the door end faces.

[0115] S257. Taking the side shooting position point as a reference, continue to control the acquisition device to move down to the next side shooting position point in the height direction according to the side shooting interval distance.

[0116] It is worth mentioning that before the drone moves down to the next side shooting position point for image acquisition, the lock state of the container corresponding to the previous position point will be analyzed, and then the corresponding downward image acquisition will be carried out. That is, when analyzing the state of the door lock with a hanging lock, this solution will also control the drone to acquire detailed images of the corresponding container. After the detailed images of the previous container are acquired, the drone can be controlled to move to the side shooting position point and then move down accordingly according to the side shooting interval distance. After reaching the next side shooting position point, image acquisition of the next-level container will be carried out, and then the corresponding lock state analysis will be carried out.

[0117] Specifically, when moving to the next side shooting position point, it can be based on the position point where the drone is currently located, and after moving down the side shooting interval distance, it will reach the next side shooting position point.

[0118] S258. Repeat the above steps of identifying the door lock structure according to the side image until the acquisition ordinal number is the preset acquisition ordinal number corresponding to the container stack, and then stop the acquisition of the side images of the target side.

[0119] Repeat the above steps of identifying the door lock structure according to the side image, that is, repeat the steps of shooting side images at the side shooting position point and analyzing the door lock of the side image. When there is a door lock structure in the side image, the lock state can be further identified, so that the management personnel can be reminded in time when the lock state is abnormal. The preset acquisition ordinal number can correspond to the stacking layers of the container stack. When the container stack has 4 layers, the preset acquisition ordinal number can be 4, so that the drone can be controlled to sequentially acquire side images of each layer of the container.

[0120] It is worth mentioning that when the drone conducts image acquisition on the other end face side, after moving to the corresponding guiding point, it can arrange the shooting heights of multiple targets from large to small to obtain an acquisition sequence, and then successively move down to the position points corresponding to the shooting heights of the corresponding targets according to the acquisition sequence to conduct image acquisition on the corresponding containers. Moreover, the acquired images can be directly determined to meet the monitoring and recognition conditions for subsequent door lock state recognition, thereby improving the efficiency of detecting the door lock state on the other side.

[0121] S26. Analyze the door lock structure of the side image, determine the side image that meets the angle recognition condition as the first target image, and perform safety state analysis on the first target image according to the angle analysis model to obtain angle analysis data.

[0122] It can be understood that in practical applications, the door lock structure of a container may include a rotary door lock combination and a door lock combination with a hanging lock. The former door lock combination generally includes a door handle and a door handle support. The box door is locked by rotating the door handle to the corresponding position of the door handle support, while the latter door lock combination generally locks the box door through a hanging lock.

[0123] Therefore, when detecting the door lock state of the former door lock combination, the closing state of the door lock structure can be determined by the included angle between the door handle and the door handle support. When detecting the door lock state of the latter door lock combination, it is necessary to judge whether the door lock state is abnormal by judging the state of the hanging lock.

[0124] Specifically, the side image that meets the angle recognition condition corresponds to the side image of the rotary door lock combination of the door lock structure. When identifying the state of this door lock structure, the closing angle between the door lock structures can be analyzed through the angle analysis model.

[0125] Among them, the angle analysis model is a model for analyzing the closing angle of a door lock combination without a hanging lock, and the angle analysis data is the data corresponding to an abnormal door lock or a normal door lock obtained after angle analysis of the door lock state.

[0126] Step S26 specifically includes steps S261 to S263, as follows:

[0127] S261. Obtain the side image that meets the monitoring and recognition condition as the recognition image, and perform hanging lock recognition on the door lock handle combination corresponding to the door lock structure in the recognition image.

[0128] It can be understood that only when the side image meets the monitoring and recognition condition, will the present solution perform door lock state recognition on the corresponding side image.

[0129] In practical applications, the contour corresponding to the door lock handle combination can be extracted, and then the sub - contours corresponding to the door handle, door handrest, and hanging lock in the extracted contour can be determined through the standard image corresponding to the door lock handle combination taken in advance. If there is no hanging lock in the door lock handle combination, that is, when it corresponds to a rotary door lock combination, the sub - contour corresponding to the hanging lock cannot be obtained. Based on this, the door lock handle combination can be identified for the hanging lock to determine the corresponding type of door lock combination.

[0130] S262, determine that the side image of the door lock handle combination without a hanging lock meets the angle recognition condition, and obtain the side image as the first target image.

[0131] The side image of the door lock handle combination without a hanging lock is the side image corresponding to the rotary door lock combination. Therefore, it can be determined as the first target image, that is, the image for detecting the door lock state of the rotary door lock combination.

[0132] S263, obtain the closing angle corresponding to each door lock handle combination in the first target image according to the angle analysis model, and perform a safety state analysis on the closing angle to obtain angle analysis data.

[0133] Among them, the closing angle is the included angle between the door handle and the door handrest in the door lock handle combination. As Figure 3 shown, when the rotary door lock combination is closed, the door handle and the door handrest are closely combined. Therefore, when determining the door lock state corresponding to this type of door lock handle combination, corresponding abnormal analysis can be performed through the closing angle corresponding to the door lock handle combination.

[0134] In some embodiments, the above - mentioned angle analysis data can be obtained through the following steps:

[0135] S2631, obtain the door handrest contour, door handle contour, and door lock rod contour connected to each door lock handle combination in the first target image.

[0136] See Figure 3 , the door lock handle combination is generally connected to the door lock rod. Therefore, when determining the closing angle corresponding to the door lock handle combination, it can be determined accordingly through the door handrest contour, door handle contour, and door lock rod contour.

[0137] S2632, determine the connection point between the door lock rod contour and the door handrest contour, connect the center point of the door handle contour and the connection point to obtain the handle angle line, and connect the center point of the door handrest contour and the connection point to obtain the reference angle line.

[0138] When determining the connection point between the door lock rod profile and the door handle rest profile, multiple profile points on the door handle profile that intersect with the door lock rod profile can be determined. Then, the extreme values of the X coordinates and the extreme values of the Y coordinates among these profile points are obtained, and the center point corresponding to the multiple profile points is determined according to the intermediate values of the extreme coordinate values, and this center point is determined as the connection point.

[0139] By connecting the center point of the door handle profile and the connection point, the diagonal direction of the door handle can be obtained. By connecting the center point of the door handle rest profile and the connection point, the inclination direction of the door handle rest can be obtained. Thus, the closing angle between the door handle and the door handle rest can be determined by the included angle between the obtained handle angle line and the reference angle line.

[0140] S2633, determine the closing angle between the handle inclination line and the reference angle line. When the closing angle is greater than or equal to the closing angle threshold, determine the corresponding door lock handle combination as an abnormal door lock combination.

[0141] When the closing angle is greater than or equal to the closing angle threshold, it indicates that the door lock handle combination may be in the open state. Therefore, it can be determined that it is an abnormal door lock combination with an abnormal door lock state. Among them, the closing angle threshold is the included angle threshold when the door handle and the door handle rest are in the closed state.

[0142] S2634, when the closing angle is less than the closing angle threshold, determine the corresponding door lock handle combination as a normal door lock combination, and obtain angle analysis data according to the abnormal door lock combination and / or the normal door lock combination.

[0143] When the closing angle is less than the closing angle threshold, it indicates that the door lock handle combination may be in the closed state, which is normal. Therefore, it can be determined that the door lock handle combination is a normal door lock combination.

[0144] Through the above method, the door lock state can be accurately analyzed, so that the management personnel can be reminded in time when the door lock is abnormal, and the safety of the goods can be improved.

[0145] S3, perform a safety state analysis on the side image that meets the detail recognition conditions according to the multi-level monitoring model to obtain detail analysis data.

[0146] Among them, the multi-level monitoring model is a model for detecting abnormal states of door lock handle combinations with hanging locks. The detail recognition condition is that the door lock structure in the side image is a door lock handle combination with a hanging lock. The detail analysis data is the data obtained after detecting the abnormal state of the door lock handle combination with a hanging lock.

[0147] When performing anomaly analysis on this type of door lock handle combination, since the side image obtained by shooting is the overall image of the container side, and the hanging lock may occupy a relatively small proportion in the image, it may not be possible to accurately identify the detailed parts of the hanging lock. Therefore, in order to more accurately identify and detect the hanging lock, this solution will control the drone to collect a magnified image of the door lock handle combination for the second time, so that the door lock handle combination in the obtained image can be clearer and the accuracy during recognition can be improved.

[0148] Based on the above embodiments, the specific implementation manner of step S3 can be:

[0149] S31. Perform door lock structure analysis on the side image, determine the side image that meets the detailed recognition condition as the second target image, and determine the corresponding detailed acquisition set of the second target image according to the multi-level monitoring model.

[0150] The detailed recognition condition is that the door lock structure is a door lock combination corresponding to a hanging lock, and the second target image is the image for detecting the door lock state of the door lock combination with a hanging lock.

[0151] It can be understood that there may be more than one door lock combination in the container door. Generally speaking, the door lock combinations on the same side are relatively close. For example, the door lock combinations corresponding to the hanging locks may also be arranged in the way of Figure 3 the left and right sides in the figure. Therefore, when performing detailed magnification acquisition on the door lock combination, the door lock combinations with relatively close positions can be collected together. For example, the two door lock combinations on the left side can be collected together, and the two door lock combinations on the right side can be collected together, so as to improve the efficiency of image acquisition. Among them, the detailed acquisition set is the set composed of door lock combinations that can be collected for image together.

[0152] Specifically, step S31 can be implemented through the following steps, specifically as follows:

[0153] S311. Obtain the side image that meets the monitoring and recognition condition as the recognition image, and perform hanging lock recognition on the door lock handle combination corresponding to the door lock structure in the recognition image.

[0154] Similarly, only when the side image meets the monitoring and recognition condition, will this solution perform door lock state recognition on the corresponding side image.

[0155] S312. Determine that the side image of the door lock handle combination with a hanging lock meets the detailed recognition condition, and obtain the side image as the second target image.

[0156] The side image of the door lock handle combination with a hanging lock is the image corresponding to the door lock combination with a hanging lock, so it can be used as the second target image.

[0157] S313. Obtain the combined contours corresponding to each door lock handle combination in the second target image according to the multi-level monitoring model, number each of the combined contours, and obtain the position numbers corresponding to each of the combined contours.

[0158] Among them, the combined contour is the total contour corresponding to the door handle support contour, the door handle contour, and the wall-mounted lock contour.

[0159] In some embodiments, when numbering the combined contours, the coordinate processing can be performed on the second target image, then the center point of each combined contour is determined as the coordinate origin, and the position where it is located is determined according to the quadrant corresponding to the center point of each combined contour in the coordinate system. It is numbered through its location to determine its corresponding position number.

[0160] Specifically, it can be determined that the position corresponding to the first quadrant can be the upper left position, the position corresponding to the second quadrant can be the upper right position, the position corresponding to the third quadrant can be the lower left position, and the position of the fourth quadrant can be the lower right position. Then, each combined contour is numbered according to the position. That is, the number of the combined contour located in the upper left position is upper left 1, the number of the combined contour located in the lower left position is lower left 1, the number of the combined contour located in the upper right position is upper right 1, and the number of the combined contour located in the lower right position is lower right 1. Thus, the relative positions corresponding to each door lock combination can be quickly determined through the numbers, so that the management personnel can quickly determine the specific positions of the door locks with abnormalities.

[0161] S314. Based on the center points of each of the combined contours, determine the combined distances between each of the combined contours, determine the door lock handle combinations with the combined distances less than the door lock distance threshold as the same set of the detail collection sets, and add corresponding collection numbers to the detail collection sets according to the position numbers.

[0162] The above-mentioned combined distance is the interval distance between the combined contours. The door lock distance threshold refers to the threshold corresponding to the interval distance when the door lock combinations are collected uniformly. The combined contours with distances less than this threshold may be contours with relatively close intervals and can be collected together for images. For example Figure 3 The two upper and lower door lock combinations on the left side, and the two upper and lower door lock combinations on the right side can be respectively used as a set of detail collection sets.

[0163] It can be understood that in order to determine the specific positions of the door lock combinations corresponding to the detail collection sets, corresponding collection numbers can be added to the detail collection sets through the position numbers of the corresponding combined contours. For example, when the detail collection set corresponds to the two upper and lower combined contours on the left side, the corresponding collection number can be upper left 1 lower left 1.

[0164] S32. Generate an outer contour box for the detail collection set, obtain the contour center point of the outer contour box, and determine the direction from the center point of the second target image to the contour center point as the adjustment direction.

[0165] When obtaining the outer contour box of the detail collection set, the X-axis extreme values and Y-axis extreme values of the coordinate points of all combined contours in the detail collection set can be obtained first. Four vertical line segments are generated based on the X-axis extreme values and Y-axis extreme values, and an outer contour box is generated through these four line segments. The outer contour box is an outer rectangular box covering all combined contours within the detail collection set, and the contour center point can be obtained through the center point of this contour box.

[0166] It can be understood that since there may be multiple detail collection sets, the drone can first fly to the corresponding position point to collect details of one detail collection set and then fly back to the original position, and then collect details of another detail collection set, so as to obtain the detailed enlarged images of each door lock combination.

[0167] Specifically, when performing detailed enlarged collection on the corresponding detail collection set, it can first move to the front of the door lock combination corresponding to the detail collection set, and then move forward to find the best collection position for collection. When moving to the front of the door lock combination corresponding to the detail collection set, the moving direction of the drone can be determined by adjusting the direction.

[0168] S33. Determine the difference distance from the center point of the second target image to the contour center point, and control the collection device to move to the first position point based on the adjustment direction and the difference distance.

[0169] Among them, the difference distance is the moving distance corresponding when the drone moves to the front of the door lock combination corresponding to the detail collection set, and the first position point is the front position point of the door lock combination corresponding to the detail collection set.

[0170] S34. Control the collection device to move forward in the detail collection direction until the interface ratio of the overall combined contour corresponding to the detail collection set in the real-time shooting interface is within the optimal shooting ratio interval, then stop the movement of the collection device and determine the corresponding position point as the detail collection point.

[0171] In practical applications, the detail collection direction can be the direction in which the drone moves forward, that is, the direction in which the drone approaches the container.

[0172] It is worth mentioning that the shooting angles of the shooting lens of the drone during top-down shooting and side shooting can be different. When shooting the container from above, the angle can be the shooting angle corresponding to the top view, and when shooting the side of the container, it can be the shooting angle corresponding to the front view. Therefore, the real-time shooting interface of the drone will always face the direction where the container is located for shooting. When the drone advances in the direction of detail collection, the center point of the real-time shooting interface can always be aligned with the center point of the overall combined contour.

[0173] Among them, the overall combined contour refers to all the combined contours corresponding to the detail collection set. When determining the interface ratio, an external rectangular frame corresponding to the overall combined contour can be generated. Specifically, 4 mutually perpendicular line segments can be generated through the extreme values of the X-axis and Y-axis corresponding to the overall combined contour to form the external rectangular frame, and then the interface ratio is determined by the ratio of the area of the external rectangular frame in the real-time shooting interface.

[0174] The optimal shooting ratio range refers to the optimal shooting ratio of the door lock combination in the real-time shooting interface. When the interface ratio of the overall combined contour corresponding to the detail collection set is within the optimal shooting ratio range, it indicates that the image ratio obtained by shooting the door lock combination corresponding to the detail collection set at the current position point will be the best. Therefore, the movement of the drone can be stopped, and the corresponding position point can be determined as the detail collection point. The detail collection point is the optimal position point for collecting magnified detail images of the door lock combination.

[0175] S35, obtain the detail image captured by the acquisition device at the detail collection point, and perform security identification on the hanging lock in the detail image to obtain detail analysis data.

[0176] The detail image is the image obtained after magnifying the collection of the door lock combination. In some embodiments, the hanging lock in the detail image can be securely identified through the following steps to obtain detail analysis data:

[0177] S35, extract the hanging lock contour corresponding to the hanging lock in each door lock handle combination in the detail image. When the hanging lock contour is in the open state, determine the corresponding door lock handle combination as an abnormal door lock combination.

[0178] It can be understood that there may be more than one door lock handle combination in the detail image. Therefore, when analyzing the door lock state, the hanging lock contours corresponding to the hanging locks in each door lock handle combination can be extracted, and the door lock state can be identified accordingly through the hanging lock contour.

[0179] It can be understood that when the hanging lock is closed, the lock rod generally engages with the lock shell. For example, when the lock rod is U-shaped, it engages with both ends of the lower lock shell when closed. Therefore, when determining the door lock state of the hanging lock, it can be judged by whether both ends of the lock rod are engaged with the lock shell. When one end of the lock rod is not engaged with the lock shell, it can be determined that it is in the open state, and the door lock combination of the corresponding container may have been opened. Therefore, the corresponding door lock handle combination can be determined as an abnormal door lock combination.

[0180] It is worth mentioning that if the door lock structure of the corresponding container is a door lock structure with a hanging lock, but the outline of the hanging lock is not recognized during identification, the corresponding door lock handle combination can also be determined as an abnormal door lock combination.

[0181] S351, when the outline of the hanging lock is in the closed state, retrieve the standard outline of the hanging lock, and compare the outline of the hanging lock with the standard outline of the hanging lock to obtain a comparison value. The standard outline of the hanging lock includes a front standard outline and a back standard outline.

[0182] When both ends of the lock rod are engaged with the lock shell, it can be determined that it is in the closed state. It can be understood that in actual applications, it may also occur that the container is opened halfway and then someone locks it again. In this case, if it is not authorized to open the container, it may also affect the safety of the goods in the container.

[0183] And the orientation of the lock surface when the container is locked generally does not change. For example, when it is locked for the first time, the front of the lock faces outward. Without external force, its orientation usually will not change the next time. If the orientation of the lock changes, it may be that someone has opened the container halfway and then locked it again. Therefore, after determining the opening and closing state of the container, this solution will also perform a secondary detection of the orientation of the container.

[0184] It can be understood that the front image and the back image of the lock are generally different. Therefore, when determining the orientation of the lock, the orientation corresponding to the current hanging lock can be determined by the comparison value between the standard outline of the hanging lock and the outline of the hanging lock.

[0185] Among them, the front standard outline refers to the outline image corresponding to the front of the lock, and the back standard outline refers to the outline image corresponding to the back of the lock.

[0186] S352, obtain the outline of the hanging lock with the comparison value greater than the similarity threshold as the secondary determination outline, and determine that the orientation of the secondary determination outline compared with the front standard outline is the positive orientation, and the orientation of the secondary determination outline compared with the back standard outline is the negative orientation.

[0187] The similarity threshold refers to the similarity value during contour comparison. If it is greater than this threshold, it indicates that the similarity between two contours is very high and they are very likely to be the same. Therefore, it can be determined that the orientation of the secondary determination contour compared with the front standard contour is the positive orientation, and the orientation of the secondary determination contour compared with the back standard contour is the negative orientation.

[0188] S353. Retrieve multiple reference container door images corresponding to the container stack. Each of the reference container door images is provided with a corresponding preset end face side and a preset acquisition sequence number.

[0189] Among them, the reference container door image refers to the image corresponding to the door of each container in the container stack. This image can be obtained by taking a picture after locking the container stack. It can be understood that since the end face side and the acquisition sequence number corresponding to the door of each container may be different, in order to determine the initial orientation of the hanging lock in the door of each container, a corresponding preset end face side and a preset acquisition sequence number can be configured for each reference container door image, so that the orientation of the hanging lock in the corresponding door structure of the corresponding container can be compared through the preset end face side and the preset acquisition sequence number.

[0190] S354. Determine the reference container door image corresponding to the detail image as the comparison container door image according to the end face side and the acquisition sequence number of the side image corresponding to the detail image.

[0191] It is worth mentioning that the comparison container door image contains all the door lock combinations, while the detail image only contains the magnified door lock combinations. Therefore, when making a comparison, the position number of each door lock structure in the comparison container door image and the position number of each door lock structure in the detail image can be determined first, and then the orientation of the hanging lock in the corresponding door lock structure can be compared through the position number.

[0192] When determining the position number of the door lock structure in the detail image, its acquisition side can be determined first, and then its corresponding upper and lower positions can be determined for numbering. For example, when the drone acquires the two door lock structures on the left side, the position number of the door lock structure at the upper position can be determined as upper left 1, and the position number of the door lock structure at the lower position can be determined as lower left 1.

[0193] S355. Determine the reference orientation corresponding to the reference hanging lock contour with the same position number as the secondary determination contour in the comparison container door image. If the orientation of the secondary determination contour is consistent with the reference orientation, determine the corresponding door lock handle combination as the normal door lock combination.

[0194] In practical applications, the corresponding benchmarks for the orientations of the reference hanging lock contours in the cabinet door image can be configured in advance. When the orientation of the secondary determination contour is consistent with the reference orientation, it indicates that the orientation of the hanging lock has not changed and it may be normal. Therefore, the corresponding door lock handle combination can be determined as a normal door lock combination.

[0195] S356, if the orientation of the secondary determination contour is inconsistent with the reference orientation, determine the corresponding door lock handle combination as an abnormal door lock combination, and obtain detailed analysis data based on the abnormal door lock combination and / or the normal door lock combination.

[0196] If the orientation of the secondary determination contour is inconsistent with the reference orientation, it indicates that the orientation of the hanging lock has changed. It may have been opened midway and is abnormal. Therefore, the corresponding door lock handle combination can be determined as an abnormal door lock combination.

[0197] Through the above method, the state of the door lock structure can be analyzed at multiple levels, thereby improving the accuracy of detecting the door lock state.

[0198] S4, generate the safety monitoring data corresponding to the container stack based on the angle analysis data or the detailed analysis data, and update the management model according to the safety monitoring data to obtain a safety monitoring model.

[0199] After obtaining the angle analysis data or the detailed analysis data, this solution will also generate the safety monitoring data corresponding to the corresponding container stack based on the analyzed data to update the management model, so that the management personnel can view the monitoring situation corresponding to each container stack through the safety monitoring model. Among them, the safety monitoring data is the door lock monitoring data corresponding to the container stack.

[0200] Based on the above embodiments, the specific implementation manner of step S4 can be:

[0201] S41, obtain multiple sub-containers corresponding to the container stack, and construct monitoring nodes corresponding to each sub-container.

[0202] Among them, the sub-container is multiple containers stacked corresponding to the container stack. It can be understood that the door lock monitoring data corresponding to each container may be different. In order to enable the management personnel to view the monitoring situation corresponding to each container, monitoring nodes corresponding to each sub-container can be constructed, and the data collected corresponding to each sub-container can be bound through the monitoring nodes in the follow-up, so that the management personnel can retrieve the collected data corresponding to the corresponding container and send it to the management personnel for viewing after clicking the corresponding node.

[0203] S42. Arrange the monitoring nodes in descending order according to the stacking layers corresponding to each sub-container, and then connect them to obtain a monitoring structure.

[0204] It can be understood that the arrangement order in the monitoring structure corresponds to the order when the drone conducts data collection.

[0205] S43. Obtain the acquisition ordinal numbers corresponding to each of the angle analysis data or the detail analysis data. According to the acquisition ordinal numbers and the arrangement order of each monitoring node, bind the monitoring nodes and the angle analysis data or the detail analysis data with the same acquisition ordinal number and arrangement order.

[0206] It can be understood that the monitoring structures corresponding to different end face sides are different. When binding the monitoring nodes and the angle analysis data or the detail analysis data with the same acquisition ordinal number and arrangement order, it is to bind the monitoring nodes on the same side and the corresponding angle analysis data or the detail analysis data. To distinguish the end face side corresponding to the monitoring structure, a label corresponding to the corresponding end face side can be added to the monitoring structure.

[0207] For example, if the first monitoring node corresponds to the topmost container, then when binding, it is also to bind the angle analysis data or the detail analysis data collected from the topmost container with the first monitoring node.

[0208] S44. Determine the monitoring nodes with the abnormal door lock combination as abnormal nodes, and determine the warning level of the abnormal nodes according to the abnormal quantity corresponding to the abnormal door lock combination.

[0209] It can be understood that the more the door lock structures with abnormalities appear, it indicates that the safety factor of the container may be lower. Therefore, the corresponding warning level can also be higher, enabling the management personnel to conduct a backcheck on the corresponding container in a timely manner and reducing the safety hazards of the goods in the container.

[0210] When determining the warning level, multiple warning levels can be set in advance, and a corresponding abnormal quantity interval can be set for each warning level. Then, determine the corresponding warning level through the abnormal quantity interval where the abnormal quantity is located.

[0211] S45. Retrieve the pixel values corresponding to the corresponding warning level to update the abnormal nodes, obtain the safety monitoring data corresponding to the container stack, and bind the safety monitoring data with the corresponding monitoring points in the management model to obtain a safety monitoring model.

[0212] In practical applications, the higher the warning level, the more prominent the corresponding pixel value can be, so that managers can observe the safety level of the corresponding container through the pixel value. For example, the pixel value corresponding to the highest warning level can be dark red, and the lower the level, the lighter the red pixel value can be.

[0213] Through the above method, containers with a low safety factor can be warned, enabling managers to make targeted treatment strategies in a timely manner.

[0214] See Figure 4 , which is a schematic structural diagram of a data processing system of a semi-high container management system provided by an embodiment of the present invention. The data processing system of the semi-high container management system includes:

[0215] A management module for constructing a management model corresponding to a semi-high container yard, where the management model includes monitoring points corresponding to multiple container stacks;

[0216] An angle module for obtaining a side image captured by a collection device at the monitoring point of the corresponding container stack, and performing a safety status analysis on the side image that meets the angle recognition condition according to an angle analysis model to obtain angle analysis data;

[0217] A detail module for performing a safety status analysis on the side image that meets the detail recognition condition according to a multi-level monitoring model to obtain detail analysis data;

[0218] A monitoring module for generating safety monitoring data corresponding to the container stack according to the angle analysis data or the detail analysis data, and updating the management model according to the safety monitoring data to obtain a safety monitoring model.

[0219] Figure 4 The device in the illustrated embodiment can correspondingly be used to execute the steps in the method embodiment shown in Figure 1 , and its implementation principle and technical effects are similar, which will not be elaborated here.

[0220] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method for a semi-high container management system, characterized in that: include: Construct a management model corresponding to the semi-high container yard, which includes monitoring points corresponding to multiple container stacks; Obtain the side image of the corresponding container stack taken by the acquisition equipment at the monitoring point, perform safety status analysis on the side image that meets the angle recognition conditions according to the angle analysis model, and obtain angle analysis data, including: Obtain a real-time shooting interface of the acquisition device at the monitoring point based on a preset overhead shooting height, extract the overhead shooting contour of the stack in the real-time shooting interface, and the center point of the overhead shooting contour of the stack corresponds to the center point of the real-time shooting interface; Determine a contour line corresponding to the end face side of the container stack in the stack overhead profile as a reference line, wherein the end face side includes a front end with a container door and a rear end without a container door; Connect the center points of the two baselines to obtain a guide line, and extend both ends of the guide line according to the overall shooting distance; Determine the direction from the center point of the guide line to the endpoint of the guide line as the guide direction, and control the acquisition device to move to the guide point corresponding to the corresponding endpoint based on the guide direction and the guide line; The end face side corresponding to the guide point is obtained as the target side, and the acquisition device is controlled to take images of the target side in the height direction according to the side shooting interval distance to obtain multiple side images, including: Taking the guide point as the reference point, control the acquisition device to move downward to the side shooting position point in the height direction according to the side shooting interval distance; Obtain a side image taken by the acquisition device at the side shooting position, extract the end face contour in the side image, and obtain four edge points corresponding to the first end face contour in the acquisition direction; Generate a recognition frame according to the edge points, and determine the image area in the side image that is within the recognition frame as the recognition area; Perform door lock recognition on the recognition area, and when there is a door lock structure in the recognition area, determine that the side image meets the monitoring and recognition conditions; When there is no door lock structure in the recognition area, it is determined that the side image does not meet the monitoring recognition condition, and the side image is deleted, and the acquisition sequence number corresponding to the side image is determined to be the target sequence number; Determine the current shooting height corresponding to the target sequence number as the target shooting height on the other end face side; Taking the side-shooting position point as a reference, continue to control the acquisition device to move downward in the height direction to the next side-shooting position point according to the side-shooting interval distance; Repeat the above steps of identifying the door lock structure according to the side image until the acquisition sequence number is the preset acquisition sequence number corresponding to the container stack, and stop acquiring the side image of the target side; Performing a door lock structure analysis on the side image, determining that the side image that meets the angle recognition condition is the first target image, and performing a safety status analysis on the first target image according to the angle analysis model to obtain angle analysis data; Perform safety status analysis on the side image that meets the detail recognition conditions according to the multi-level monitoring model to obtain detail analysis data; Safety monitoring data corresponding to the container stacking is generated according to the angle analysis data or the detail analysis data, and the management model is updated according to the safety monitoring data to obtain a safety monitoring model.

2. The method according to claim 1, characterized in that Performing a door lock structure analysis on the side image, determining that the side image that meets the angle recognition condition is a first target image, and performing a safety status analysis on the first target image according to an angle analysis model to obtain angle analysis data, including: A side image satisfying the monitoring and identification condition is obtained as an identification image, and a hanging lock identification is performed on a door lock handle combination corresponding to the door lock structure in the identification image; Determine that the side image of the door lock handle combination without a hanging lock meets the angle recognition condition, and obtain the side image as the first target image; The closing angle corresponding to each door lock handle combination in the first target image is obtained according to the angle analysis model, and the safety status analysis of the closing angle is performed to obtain angle analysis data.

3. The method according to claim 2, characterized in that Acquiring the closing angle corresponding to each door lock handle combination in the first target image according to the angle analysis model, and performing safety status analysis on the closing angle to obtain angle analysis data, including: Acquire the outline of the door handle support and the door handle in each of the door lock handle assemblies in the first target image, and the outline of the door lock rod connected to each of the door lock handle assemblies; Determine the connection point between the door lock rod contour and the door handle support contour, connect the center point of the door handle contour and the connection point to obtain a handle angle line, and connect the center point of the door handle support contour and the connection point to obtain a reference angle line; Determine a closing angle between the handle tilt line and the reference angle line, and when the closing angle is greater than or equal to a closing angle threshold, determine that the corresponding door lock handle combination is an abnormal door lock combination; When the closing angle is less than the closing angle threshold, the corresponding door lock handle combination is determined to be a normal door lock combination, and angle analysis data is obtained according to the abnormal door lock combination and / or the normal door lock combination.

4. The method according to claim 1, characterized in that: According to the multi-level monitoring model, the safety status of the side image that meets the detail recognition conditions is analyzed to obtain the detail analysis data, including: Performing a door lock structure analysis on the side image, determining that the side image that meets the detail recognition condition is a second target image, and determining a detail collection set corresponding to the second target image according to the multi-level monitoring model; Generate an outer contour frame of the detail collection set, obtain a contour center point of the outer contour frame, and determine a direction from a center point of the second target image to the contour center point as an adjustment direction; Determine a phase difference distance between a center point of the second target image and a center point of the contour, and control the acquisition device to move to a first position point based on the adjustment direction and the phase difference distance; Control the acquisition device to move forward in the detail acquisition direction until the interface proportion of the overall combined contour corresponding to the detail acquisition set in the real-time shooting interface is within the optimal shooting proportion interval, and stop the movement of the acquisition device, and determine the corresponding position point as the detail acquisition point; The detail image captured by the acquisition device at the detail acquisition point is acquired, and the hanging lock in the detail image is securely identified to obtain detail analysis data.

5. The method according to claim 4, characterized in that Performing a door lock structure analysis on the side image, determining that the side image that meets the detail recognition condition is a second target image, and determining a detail collection set corresponding to the second target image according to the multi-level monitoring model, including: A side image satisfying the monitoring and identification condition is obtained as an identification image, and a hanging lock identification is performed on a door lock handle combination corresponding to the door lock structure in the identification image; Determine that the side image of the door lock handle combination with the hanging lock meets the detail recognition condition, and obtain the side image as the second target image; Acquire the combined contours corresponding to each door lock handle combination in the second target image according to the multi-level monitoring model, number each of the combined contours, and obtain the position number corresponding to each of the combined contours; Based on the center point of each of the combined contours, the combined distance between each of the combined contours is determined, the door lock handle combinations whose combined distance is less than the door lock distance threshold are determined to be the same group of the detail collection set, and a corresponding collection number is added to the detail collection set according to the position number.

6. The method according to claim 5, characterized in that Acquiring a detail image captured by the acquisition device at the detail acquisition point, performing security identification on the hanging lock in the detail image, and obtaining detail analysis data, including: Extracting the hanging lock outline corresponding to the hanging lock in each door lock handle combination in the detail image, and when the hanging lock outline is in an open state, determining that the corresponding door lock handle combination is an abnormal door lock combination; When the hanging lock profile is in a closed state, calling a standard hanging lock profile, comparing the hanging lock profile with the standard hanging lock profile to obtain a comparison value, wherein the standard hanging lock profile includes a front standard profile and a back standard profile; The locket contour having the comparison value greater than the similarity threshold is obtained as the secondary determination contour, and the orientation of the secondary determination contour compared with the front standard contour is determined as the positive orientation, and the orientation of the secondary determination contour compared with the rear standard contour is determined as the rear orientation; Retrieving a plurality of reference door images corresponding to the container stack, each of the reference door images being provided with a corresponding preset end face side and a preset acquisition sequence number; Determine, according to the end face side and the acquisition sequence number of the side image corresponding to the detail image, a reference door image corresponding to the detail image as a comparison door image; Determine a reference orientation corresponding to a reference hanging lock contour having the same position number as the second determination contour in the comparison door image, and if the orientation of the second determination contour is consistent with the reference orientation, determine that the corresponding door lock handle combination is a normal door lock combination; If the orientation of the secondary determination profile is inconsistent with the reference orientation, the corresponding door lock handle combination is determined to be an abnormal door lock combination, and detailed analysis data is obtained according to the abnormal door lock combination and / or the normal door lock combination.

7. The method according to claim 3 or 6, characterized in that: Generating safety monitoring data corresponding to the container stacking according to the angle analysis data or the detail analysis data, and updating the management model according to the safety monitoring data to obtain a safety monitoring model, including: Acquire multiple sub-containers corresponding to the container stack, and construct monitoring nodes corresponding to each of the sub-containers; According to the number of stacking layers corresponding to each of the sub-containers, the monitoring nodes are arranged from large to small and then connected to obtain a monitoring structure; Obtaining a collection sequence number corresponding to each of the angle analysis data or the detail analysis data, and binding the monitoring nodes and the angle analysis data or the detail analysis data having the same collection sequence number and arrangement order according to the collection sequence number and the arrangement order of each of the monitoring nodes; Determine the monitoring node where the abnormal door lock combination exists as an abnormal node, and determine the warning level of the abnormal node according to the number of abnormalities corresponding to the abnormal door lock combination; The pixel value corresponding to the corresponding warning level is retrieved to update the abnormal node, and the safety monitoring data corresponding to the container stack is obtained, and the safety monitoring data is bound to the corresponding monitoring point in the management model to obtain a safety monitoring model.

8. A system corresponding to the data processing method of the semi-high container management system according to claim 1, characterized in that: include: A management module, used to construct a management model corresponding to a semi-high container yard, wherein the management model includes monitoring points corresponding to a plurality of container stacks; An angle module is used to obtain the side image of the corresponding container stack taken by the acquisition device at the monitoring point, and perform safety status analysis on the side image that meets the angle recognition condition according to the angle analysis model to obtain angle analysis data, including: Obtain a real-time shooting interface of the acquisition device at the monitoring point based on a preset overhead shooting height, extract the overhead shooting contour of the stack in the real-time shooting interface, and the center point of the overhead shooting contour of the stack corresponds to the center point of the real-time shooting interface; Determine a contour line corresponding to the end face side of the container stack in the stack overhead profile as a reference line, wherein the end face side includes a front end with a container door and a rear end without a container door; Connect the center points of the two baselines to obtain a guide line, and extend both ends of the guide line according to the overall shooting distance; Determine the direction from the center point of the guide line to the endpoint of the guide line as the guide direction, and control the acquisition device to move to the guide point corresponding to the corresponding endpoint based on the guide direction and the guide line; The end face side corresponding to the guide point is obtained as the target side, and the acquisition device is controlled to take images of the target side in the height direction according to the side shooting interval distance to obtain multiple side images, including: Taking the guide point as the reference point, control the acquisition device to move downward to the side shooting position point in the height direction according to the side shooting interval distance; Obtain a side image taken by the acquisition device at the side shooting position, extract the end face contour in the side image, and obtain four edge points corresponding to the first end face contour in the acquisition direction; Generate a recognition frame according to the edge points, and determine the image area in the side image that is within the recognition frame as the recognition area; Perform door lock recognition on the recognition area, and when there is a door lock structure in the recognition area, determine that the side image meets the monitoring and recognition conditions; When there is no door lock structure in the recognition area, it is determined that the side image does not meet the monitoring recognition condition, and the side image is deleted, and the acquisition sequence number corresponding to the side image is determined to be the target sequence number; Determine the current shooting height corresponding to the target sequence number as the target shooting height on the other end face side; Taking the side-shooting position point as a reference, continue to control the acquisition device to move downward in the height direction to the next side-shooting position point according to the side-shooting interval distance; Repeat the above steps of identifying the door lock structure according to the side image until the acquisition sequence number is the preset acquisition sequence number corresponding to the container stack, and stop acquiring the side image of the target side; Performing a door lock structure analysis on the side image, determining that the side image that meets the angle recognition condition is the first target image, and performing a safety status analysis on the first target image according to the angle analysis model to obtain angle analysis data; A detail module is used to perform safety status analysis on the side image that meets the detail recognition conditions according to the multi-level monitoring model to obtain detail analysis data; A monitoring module is used to generate safety monitoring data corresponding to the container stacking according to the angle analysis data or the detail analysis data, and to update the management model according to the safety monitoring data to obtain a safety monitoring model.

Images