Method and device for determining personnel in yard bridge, electronic equipment and storage medium

Abstract

The invention relates to the field of visual processing, in particular to a method and device for determining personnel in a field bridge, electronic equipment and a storage medium, and the method comprises the steps: obtaining first tracking trajectory information of each personnel in a first image in a monitoring video of the field bridge, and obtaining first personnel information corresponding to each personnel in a second image in the monitoring video; matching each piece of first personnel information with each piece of first tracking trajectory information to obtain second tracking trajectory information, third tracking trajectory information and second personnel information; according to the first position parameter and the second position parameter of the first person, determining the variable quantity of the first person on the field bridge; according to the second position parameter of the second person, determining the variable quantity of the second person on the field bridge, and determining the variable quantity of the third person on the field bridge according to the first position parameter of the third person; and determining the total number of personnel on the yard bridge according to the first personnel variation, the second personnel variation and the third personnel variation. According to the invention, the determination accuracy of the number of personnel in the field bridge is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual processing, and in particular to a method and device for determining personnel in a yard crane, electronic equipment and a storage medium. BACKGROUND

[0002] With the acceleration of the construction of smart ports and green ports, the traditional container terminal automation renovation work has entered the fast lane, and the yard crane automation renovation technology has achieved new breakthroughs and has been widely applied in many ports. However, the current renovated yard crane lacks the function of automatically determining whether there is operating personnel on the crane during automatic operation, which poses a great safety hazard. If there is operating personnel on the ladder when the yard crane is automatically driving, a falling accident is likely to occur. Therefore, it is necessary to count the personnel entering and leaving the yard crane, so that the yard crane can determine the number of personnel on the yard crane before automatic operation, and if there is anyone, the automatic operation instruction is temporarily suspended.

[0003] In the example technology, the yard crane is monitored, and the personnel in the monitored image are determined for face information, so as to determine the number of personnel in the yard crane.

[0004] However, the above is to identify the face information of personnel in the current frame image, and the current frame image may miss personnel, resulting in inaccurate determination of the number of personnel in the yard crane, that is, the accuracy of determining the number of personnel in the yard crane is poor. SUMMARY

[0005] Therefore, the present application provides a method and device for determining personnel in a yard crane, electronic equipment and a storage medium, which solves the technical problem of poor accuracy of determining the number of personnel in the yard crane.

[0006] As a first aspect of the present application, a method for determining personnel in a yard crane is provided, comprising:

[0007] Obtaining first tracking trajectory information of each personnel in a first image in a monitoring video of a yard crane, and obtaining first personnel information corresponding to each personnel in a second image in the monitoring video, the first image being a previous frame image of the second image, the first tracking trajectory information including a first position parameter of personnel in the first image, and the first personnel information including a second position parameter of personnel in the yard crane in the second image;

[0008] Matching each first personnel information and each first tracking trajectory information to obtain second tracking trajectory information, third tracking trajectory information and second personnel information, wherein the second tracking trajectory information is first tracking trajectory information matched with first personnel information, the third tracking trajectory information is first tracking trajectory information not matched with first personnel information, and the second personnel information is first personnel information not matched with first tracking trajectory information.

[0009] determine a first personnel change amount on the yard crane according to a first position parameter of the first personnel corresponding to the second tracking trajectory information and a second position parameter of the first personnel contained in the first personnel information matched by the second tracking trajectory information;

[0010] determine a second personnel change amount on the yard crane according to a second position parameter of the second personnel corresponding to the second personnel information, and determine a third personnel change amount on the yard crane according to a first position parameter of the third personnel corresponding to the third tracking trajectory information;

[0011] determine a total number of personnel on the yard crane according to the first personnel change amount, the second personnel change amount and the third personnel change amount.

[0012] In a possible implementation manner, the matching of each of the first personnel information and each of the first tracking trajectory information comprises:

[0013] predict a second detection frame parameter of the fourth personnel in the second image according to a first detection frame parameter and a first speed parameter of the fourth personnel corresponding to the first tracking trajectory information, and determine a motion feature similarity between the second detection frame parameter and a third detection frame parameter in each of the first personnel information;

[0014] determine an appearance feature similarity between an appearance feature parameter of the fourth personnel in the first tracking trajectory information and an appearance feature parameter of personnel in each of the first personnel information;

[0015] construct a cost matrix according to each of the motion feature similarities and each of the appearance feature similarities, and match each of the first personnel information and each of the first tracking trajectory information according to the cost matrix.

[0016] In a possible implementation manner, the first personnel information comprises an appearance feature parameter of the fourth personnel, and the obtaining of the first personnel information corresponding to each of the personnel in the second image in the monitoring video comprises:

[0017] obtaining a personnel image of the fourth personnel in the second image according to a second detection frame parameter of the fourth personnel in the second image;

[0018] inputting the personnel image into an appearance feature extraction model to obtain the appearance feature parameter of the fourth personnel.

[0019] In a possible implementation manner, the obtaining of the first personnel information corresponding to each of the personnel in the second image in the monitoring video comprises:

[0020] Initialize pixel values of the second image to obtain a processing image, and detect the personnel in the second image to obtain a plurality of detection frame images, the detection frame image being a second image containing a detection frame of a personnel;

[0021] According to the region where each pixel point in the second image is located, each pixel point in the processing image is valued to obtain a first mask image, and each pixel point in the detection frame image is valued according to the detection frame in the detection frame image to obtain a second mask image;

[0022] Perform XOR operation on pixel values of the pixel points at the same position in the first mask image and each second mask image to obtain the number of target pixel points corresponding to the second mask image, the pixel value of the target pixel point in the first mask image being different from the pixel value in the second mask image;

[0023] According to the ratio between the number of target pixel points corresponding to each second mask image and the total number of pixel points in the second mask image, determine the second position parameter corresponding to the personnel corresponding to each second mask image.

[0024] In a possible implementation manner, the method further includes: obtaining a historical number of times associated with a track identifier corresponding to the first tracking trajectory information that is not matched with the first personnel information, the historical number of times being used to indicate a number of matching failures of the first tracking trajectory information that is not matched with the first personnel information;

[0025] Update the historical number of times to obtain a target number of times,

[0026] When the target number of times does not reach a set number of times, determine the first tracking trajectory information that is not matched with the first personnel information as the third tracking trajectory information;

[0027] When the target number of times reaches the set number of times, delete the first tracking trajectory information that is not matched with the first personnel information.

[0028] In a possible implementation manner, the determining the first personnel change amount on the field bridge according to the first position parameter of the first personnel corresponding to the second tracking trajectory information and the second position parameter contained in the first personnel information matched by the second tracking trajectory information includes:

[0029] Compare the first position parameter with the second position parameter to obtain a first comparison result;

[0030] According to a first comparison result of each of the first persons, a first person quantity correction value corresponding to each of the first persons is determined, wherein the first comparison result indicates that the first position parameter is equal to the second position parameter, the person quantity correction value is 0, the first comparison result indicates that the first position parameter is greater than the second position parameter, the person quantity correction value is -1, and the first comparison result indicates that the first position parameter is less than the second position parameter, and the person quantity correction value is 1.

[0031] According to the person quantity correction value of each of the first persons, a first person change amount is determined.

[0032] In a possible implementation manner, after the matching of each of the first person information and each of the first tracking trajectory information, the method further includes:

[0033] A second speed parameter corresponding to each of the first persons in the first person information is obtained.

[0034] According to the first person information and the corresponding second speed parameter, tracking trajectory information of the persons in the second image is determined.

[0035] As a second aspect of the present application, the present application provides a determination device of persons in a yard bridge, comprising:

[0036] An acquisition module is configured to acquire first tracking trajectory information of each person in a first image of a monitoring video of a yard bridge, and acquire first person information corresponding to each person in a second image of the monitoring video, the first image being a previous frame image of the second image, the first tracking trajectory information including a first position parameter of a person in the first image, and the first person information including a second position parameter of the person in the second image in the yard bridge.

[0037] A matching module is configured to match each of the first person information and each of the first tracking trajectory information to obtain second tracking trajectory information, third tracking trajectory information, and second person information, wherein the second tracking trajectory information is first tracking trajectory information matched with first person information, the third tracking trajectory information is first tracking trajectory information not matched with first person information, and the second person information is first person information not matched with first tracking trajectory information.

[0038] The first determining module is configured to determine a first personnel change amount on the yard crane according to a first position parameter of a first person corresponding to the second tracking trajectory information and a second position parameter contained in first person information matched by the second tracking trajectory information, determine a second personnel change amount on the yard crane according to a second position parameter of a second person corresponding to the second person information, and determine a third personnel change amount on the yard crane according to a first position parameter of a third person corresponding to the third tracking trajectory information.

[0039] The third determining module is configured to determine a total number of persons on the yard crane according to the first personnel change amount, the second personnel change amount and the third personnel change amount.

[0040] As a third aspect of the present application, the present application further provides an electronic device comprising a memory and a processor, wherein,

[0041] The memory is connected with the processor and is configured to store a program.

[0042] The processor is configured to realize the method for determining persons in a yard crane by running the program in the memory.

[0043] As a fourth aspect of the present application, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is run by a processor to realize the method for determining persons in a yard crane.

[0044] The application provides a method and device for determining the number of people in a yard bridge, electronic equipment and a storage medium. The method for determining the number of people in the yard bridge comprises the following steps: obtaining first tracking trajectory information of each person in a previous frame image in a monitoring video of the yard bridge and first person information of each person in a current frame image, matching each first person information and each second tracking trajectory information to obtain second tracking trajectory information matched with the first person information, third tracking trajectory information that fails to be matched successfully, and second person information that fails to be matched successfully, determining a first person change in the yard bridge based on first position parameters and second position parameters of the first person corresponding to the second tracking trajectory information, determining a second person change in the yard bridge based on second position parameters of the second person corresponding to the second person information, and determining a third person change in the yard bridge based on first position information of the third person corresponding to the third tracking trajectory information, and determining the total number of people in the yard bridge based on the first person change, the second person change and the third person change. In the application, the tracking trajectory information matched successfully, the tracking trajectory information that fails to be matched successfully, and the person information that fails to be matched successfully are determined through the previous and subsequent frame images of the yard bridge monitoring video, so as to determine the person change corresponding to different matching results. Even if there is a person missing in the current frame image, the missing person can be supplemented through the tracking information that fails to be matched successfully, so that the number of people in the yard bridge is accurate, thereby improving the accuracy of determining the number of people in the yard bridge. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0046] Figure 1 Fig. 1 shows a flowchart of a method for determining the number of people in a yard bridge according to an embodiment of the application;

[0047] Figure 2 Fig. 4 shows a monitoring schematic diagram of an access control monitoring camera according to an embodiment of the application;

[0048] Figure 3 Fig. 5 shows a flowchart of a method for determining the number of people in a yard bridge according to another embodiment of the application;

[0049] Figure 4 Fig. 6 shows a flowchart of a method for determining the number of people in a yard bridge according to still another embodiment of the application;

[0050] Figure 5Fig. 1 shows a schematic diagram of image labeling in a monitoring video according to an embodiment of the present application;

[0051] Figure 6 Fig. 4 shows a schematic diagram of a fourth flow of a method for determining a person in a yard crane according to an embodiment of the present application;

[0052] Figure 7 Fig. 5 shows a structural block diagram of a device for determining a person in a yard crane according to an embodiment of the present application;

[0053] Figure 8 Fig. 6 shows a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the description of the present application, the meaning of "a plurality of" is at least two, for example two, three, etc., unless otherwise explicitly and specifically limited. All directional indications (such as upper, lower, left, right, front, back, top, bottom, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components, and if the specific posture changes, the directional indications will also change accordingly. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0055] In addition, the reference to "embodiments" in this document means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean that it refers to the same embodiment, nor is it independent or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0056] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0057] Summary of the application

[0058] With the acceleration of the construction of smart port and green port, the automation transformation of traditional container terminal enters the fast lane, and the automation transformation technology of the gantry crane realizes new breakthroughs and is widely applied in many ports. However, the current transformed gantry crane lacks the function of automatically judging whether there is an operator on the gantry crane during automatic operation, which poses a great safety hazard. If there is an operator on the gantry crane when the gantry crane is automatically driving, a falling accident is likely to occur. Therefore, it is necessary to count the personnel entering and leaving the gantry crane, so that the gantry crane can first determine the number of personnel on the gantry crane before automatic operation, and if there is anyone, the automatic operation instruction is temporarily suspended.

[0059] In the example technology, the gantry crane is monitored, and the personnel in the monitored image are judged for face information, so as to determine the number of personnel in the gantry crane.

[0060] However, the above is to identify the face information of the personnel in the current frame image, and the current frame image may miss some personnel, resulting in inaccurate determination of the number of personnel in the gantry crane, that is, the accuracy of determining the number of personnel in the gantry crane is poor.

[0061] Therefore, the present application provides a method and device for determining the number of personnel in a gantry crane, an electronic device and a storage medium. The method for determining the number of personnel in the gantry crane comprises obtaining first tracking trajectory information of each personnel in a previous frame image in a monitoring video of the gantry crane and first personnel information of each personnel in a current frame image, matching each first personnel information and each second tracking trajectory information to obtain second tracking trajectory information matched with the first personnel information, third tracking trajectory information that fails to be matched successfully, and second personnel information that fails to be matched successfully, determining a first personnel change quantity on the gantry crane based on a first position parameter and a second position parameter of a first personnel corresponding to the second tracking trajectory information, determining a second personnel change quantity on the gantry crane based on a second position parameter of a second personnel corresponding to the second personnel information, and determining a third personnel change quantity on the gantry crane based on a first position information of a third personnel corresponding to the third tracking trajectory information, and determining a total number of personnel on the gantry crane based on the first personnel change quantity, the second personnel change quantity and the third personnel change quantity. In the present application, the previous frame image and the current frame image of the gantry crane monitoring video are used to determine the tracking trajectory information that is matched successfully, the tracking trajectory information that fails to be matched successfully, and the personnel information that fails to be matched successfully, so as to determine the personnel change quantity corresponding to different matching results. Even if there is a personnel missing in the current frame image, the personnel missing can be supplemented by the tracking information that fails to be matched successfully and the personnel information that fails to be matched successfully, so as to ensure the accuracy of the number of personnel in the gantry crane, thereby improving the accuracy of determining the number of personnel in the gantry crane.

[0062] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0063] Exemplary methods

[0064] Figure 1 As shown in the figure, the method for determining personnel in a yard crane provided by the present application comprises the following steps. Figure 1 As shown in the figure, the method for determining personnel in a yard crane can specifically comprise the following steps:

[0065] In step S110, first tracking trajectory information of each personnel in a first image in a monitoring video of a yard crane is acquired, and first personnel information corresponding to each personnel in a second image in the monitoring video is acquired. The first image is a previous frame image of the second image. The first tracking trajectory information comprises a first position parameter of the personnel in the first image. The first personnel information comprises a second position parameter of the personnel in the yard crane in the second image.

[0066] In the embodiment, the execution subject is a personnel determination device in a yard crane. For the convenience of description, the device is used to represent the personnel determination device in the yard crane in the following description. The device can be a server or any terminal device with personnel determination function in a yard crane.

[0067] A yard crane is a professional container yard mechanical equipment. When the yard crane needs to work, a monitoring device is used to monitor the corresponding area of the yard crane. Referring to Figure 2 The field of view of the access control monitoring camera includes an access control platform and a yard crane ladder. The yard crane ladder is a device installed on the yard crane, which is used to facilitate the operation personnel to go up and down the yard crane.

[0068] The access control monitoring camera obtains a monitoring video after monitoring, and then sends the monitoring video to the device. The device detects the personnel in the monitoring video, so as to obtain personnel information corresponding to each personnel in a second image of a current frame. The personnel information is defined as first personnel information. The first personnel information comprises a second position parameter of the personnel in the yard crane in the second image. The second position parameter can be a numerical value, which represents whether the personnel is on the yard crane. For example, when the second position parameter is 1, it represents that the personnel corresponding to the first personnel information is on the yard crane; when the second position parameter is 0, it represents that the personnel corresponding to the first personnel information is under the yard crane. The device detects the position of the personnel in the second image. When the position is on the yard crane, the second position parameter of the personnel is 1; when the position is under the yard crane, the second position parameter of the personnel is 0.

[0069] Further, the device also needs to obtain first tracking trajectory information of each person in the first image, the first image being a previous frame image of the second image, and the first tracking trajectory information containing a first position parameter of the person in the first image, the first position parameter being 1, indicating that the person tracked by the first tracking trajectory information is on the yard bridge; and the second position parameter being 0, indicating that the person tracked by the first tracking trajectory information is under the yard bridge.

[0070] In step S120, the first personnel information and the first tracking trajectory information are matched to obtain second tracking trajectory information, third tracking trajectory information and second personnel information, wherein the second tracking trajectory information is the first tracking trajectory information matched with the first personnel information, the third tracking trajectory information is the first tracking trajectory information not matched with the first personnel information, and the second personnel information is the first personnel information not matched with the first tracking trajectory information. The first tracking trajectory information is obtained by the device last time when determining the position parameter of the person in the previous frame image.

[0071] After obtaining the first tracking trajectory information of each person in the first image and the first personnel information of each person in the second image, the device matches the first personnel information and the first tracking trajectory information to obtain the second tracking trajectory information, the third tracking trajectory information and the second personnel information. The second tracking trajectory information is the first tracking trajectory information matched with the first personnel information, the third tracking trajectory information is the first tracking trajectory information not matched with the first personnel information, and the second personnel information is the first personnel information not matched with the first tracking trajectory information. For example, the first tracking trajectory information contains a detection box parameter of the person, and the detection box parameter can be a first image coordinate of the detection box containing the person; the first personnel information also contains a detection box parameter of the person, and the detection box parameter can be a second image coordinate of the detection box containing the person; if the detection box represented by the first image coordinate and the detection box represented by the second image coordinate have an overlapping area, it can be determined that the first personnel information is matched with the first tracking trajectory information; if the detection box in the first personnel information and the detection boxes of all the first tracking trajectory information have no overlapping area, the first personnel information is not matched with the first tracking trajectory information; if the detection box in the first tracking trajectory information and the detection boxes of all the first personnel information have no overlapping area, the first tracking trajectory information is not matched with the first personnel information.

[0072] In step S130, according to the first position parameter of the first personnel corresponding to the second tracking trajectory information and the second position parameter contained in the first personnel information matched by the first personnel in the second tracking trajectory information, the first personnel change amount on the yard bridge is determined.

[0073] After determining the second tracking trajectory information, the third tracking trajectory information and the second personnel information, the personnel change amount of each case is determined according to the case.

[0074] In an example, the tracked person in the second tracking trajectory information is defined as a first person, the first person has a first position parameter in the second tracking trajectory information, and the first person also has a second position parameter in the first person information matched by the second tracking trajectory information, that is, the first person has a first position parameter in the previous frame image and has a second position parameter in the current frame image.

[0075] If the first position parameter indicates that the first person is located under the yard crane, it is further determined whether the second position parameter is located under the yard crane. If the second position parameter indicates that the first person is located under the yard crane, it is represented that the first person passes or stays under the yard crane in the previous frame and the current frame, and the number of persons is unchanged, that is, the number of person change quantity in the second tracking trajectory information is 0. If the second position parameter indicates that the first person is located above the yard crane, the first person is located under the yard crane in the previous frame, but the first person is located above the yard crane in the current frame, and the number of persons is increased by 1, that is, the number of person change quantity in the second tracking trajectory information is 1.

[0076] If the first position parameter indicates that the first person is located above the yard crane, it is further determined whether the second position parameter is located under the yard crane. If the second position parameter indicates that the first person is located above the yard crane, it is represented that the first person is above the yard crane in the previous frame and the current frame, and the number of persons is unchanged, that is, the number of person change quantity in the second tracking trajectory information is 0. If the second position parameter indicates that the first person is located above the yard crane, the first person is located above the yard crane in the previous frame, but the first person is located under the yard crane in the current frame, and the number of persons is decreased by 1, that is, the number of person change quantity in the second tracking trajectory information is -1. Each second tracking trajectory information corresponds to a number of person change quantity, and the sum of each number of person change quantity is the first person change quantity.

[0077] In another example, a first comparison result is compared between the first position parameter and the second position parameter. According to the first comparison result of each first person, a first person number correction value corresponding to each first person is determined, wherein the first comparison result indicates that the first position parameter is equal to the second position parameter, and the person number correction value is 0. The first comparison result indicates that the first position parameter is greater than the second position parameter, and the person number correction value is -1. The first comparison result indicates that the first position parameter is less than the second position parameter, and the person number correction value is 1. According to the person number correction value of each first person, the first person change quantity is determined.

[0078] For example, if the first person has L (L is the second position parameter) = 0 in the current frame and L' (L' is the first position parameter) = 0 in the previous frame, it indicates that the first person is in the bridge in the previous frame and the current frame, and the state analysis is that the first person is passing or staying under the machine, the first person is not counted in the personnel quantity change, and the personnel quantity does not change; if L = 1 in the current frame and L' = 1 in the previous frame, it indicates that the first person is in the bridge in the previous frame and the current frame, and the state analysis is that the first person is walking or staying on the ladder, the first person is not counted in the personnel quantity change, that is, the personnel quantity on the bridge does not change; if L = 1 in the current frame and L' = 0 in the previous frame, it indicates that the first person is under the bridge in the previous frame and on the machine in the current frame, and the state analysis is that the first person is getting on the access control platform, and the first person is counted in the personnel quantity change, that is, the personnel quantity is increased by 1; if L = 0 in the current frame and L' = 1 in the previous frame, it indicates that the first person is on the machine in the previous frame and under the machine in the current frame, and the state analysis is that the first person is getting off the access control platform, and the first person is counted in the personnel quantity change, that is, the personnel quantity is reduced by 1.

[0079] In step S140, the second personnel change on the bridge is determined according to the second position parameter of the second person corresponding to the second personnel information, and the third personnel change on the bridge is determined according to the first position parameter of the third person corresponding to the third tracking trajectory information.

[0080] After determining the first personnel change, the device determines the second personnel change on the bridge based on the second position parameter of the second person corresponding to the second personnel information.

[0081] For example, the second personnel information is the first personnel information that is not matched with the first tracking trajectory information, that is, the second personnel information is a new trajectory of the second image in the current frame. Since the second personnel information is a new trajectory, the second personnel information does not have a position parameter in the previous frame image and only has a position parameter in the current frame image, that is, the second personnel only has a second position parameter. If the second position parameter is 0, it indicates that the second personnel enters the monitoring picture under the bridge, the second personnel does not enter the bridge, and the personnel change corresponding to the second personnel information is 0; when the second position parameter is 1, it indicates that the second personnel enters the monitoring picture on the bridge, that is, the second personnel gets off the bridge ladder, and the personnel quantity does not change, that is, the personnel change corresponding to the second personnel information is 0. The sum of the personnel changes corresponding to all the second personnel information is the second personnel change.

[0082] After determining the second personnel change, the device determines the third personnel change on the bridge based on the first position parameter of the third person corresponding to the third tracking trajectory information.

[0083] Exemplarily, the third tracking trajectory information is first tracking trajectory information which is not matched with the first personnel information, for the first tracking trajectory information which is not matched, the third personnel has the first position parameter in the previous frame but has no position parameter in the current frame, thus, only the first position parameter of the third personnel in the third tracking trajectory information needs to be analyzed. If the first position parameter of the third personnel is 0, it indicates that the third personnel leaves the monitoring picture under the platform bridge, the number of personnel does not change, that is, the personnel change quantity corresponding to the third tracking trajectory information is 0; if the first position parameter of the third personnel is 1, it indicates that the third personnel leaves the monitoring picture on the platform bridge, that is, the third personnel climbs the ladder after getting off the platform bridge, the number of personnel on the platform bridge does not change, that is, the personnel change quantity corresponding to the third tracking trajectory information is 0. The sum of the personnel change quantities of all the third tracking trajectory information is the third personnel change quantity.

[0084] In step S150, the total number of personnel on the platform bridge is determined according to the first personnel change quantity, the second personnel change quantity and the third personnel change quantity.

[0085] After obtaining the first personnel change quantity, the second personnel change quantity and the third personnel change quantity, the device superimposes the first personnel change quantity, the second personnel change quantity and the third personnel change quantity to obtain a total personnel change quantity. The device stores the number of personnel on the platform bridge, and the sum of the number of personnel and the total personnel change quantity is the current number of personnel on the platform bridge.

[0086] In the embodiment, the first tracking trajectory information of each personnel in the previous frame image in the monitoring video of the platform bridge and the first personnel information of each personnel in the current frame image are obtained, each first personnel information and each second tracking trajectory information are matched to obtain the second tracking trajectory information matched with the first personnel information, the third tracking trajectory information which is not matched successfully and the second personnel information which is not matched successfully, the first personnel change quantity on the platform bridge is determined based on the first position parameter and the second position parameter of the first personnel corresponding to the second tracking trajectory information, the second personnel change quantity on the platform bridge is determined based on the second position parameter of the second personnel corresponding to the second personnel information, and the third personnel change quantity on the platform bridge is determined based on the first position information of the third personnel corresponding to the third tracking trajectory information, so as to determine the total number of personnel on the platform bridge based on the first personnel change quantity, the second personnel change quantity and the third personnel change quantity. In the embodiment, the previous frame image and the current frame image of the platform bridge monitoring video are used to determine the tracking trajectory information which is matched successfully, the tracking trajectory information which is not matched successfully and the personnel information which is not matched successfully, so as to determine the personnel change quantity corresponding to different matching results. Even if there is personnel omission in the current frame image, the personnel quantity can be checked and supplemented through the tracking information which is not matched successfully, so as to ensure that the number of personnel on the platform bridge is accurate, thereby improving the determination accuracy of the number of personnel on the platform bridge.

[0087] Referring toFigure 3 , Figure 3 This is a flow chart of a method for determining personnel in a field crane provided by this application, based on the following Figure 1 In the illustrated embodiment, step S120 includes:

[0088] Step S310: Predict the second detection frame parameters of the fourth person in the second image based on the first detection frame parameters and the first speed parameters of the fourth person corresponding to the first tracking trajectory information, and determine the motion feature similarity between the second detection frame parameters and the third detection frame parameters in each first person information.

[0089] In this embodiment, the first person information is the information corresponding to each person in the K-th frame image in the surveillance video. Each first person information P includes a person detection frame B, a person appearance feature F, and a current frame person position L, where L is the second position parameter.

[0090] The first tracking trajectory information is the tracking trajectory information in the k-1 frame image, that is, the tracking trajectory of the k-1 frame. Where J represents the total number of tracked trajectories, and j represents the trajectory sequence number. Each piece of trajectory information T includes the trajectory frame B′ (first detection frame parameter), trajectory motion speed V′ (first speed parameter), trajectory appearance feature F′ (appearance feature parameter), and trajectory position L′ (first position parameter). The speed V′ includes the speeds of the four parameter directions within the frame B′.

[0091] The device passes through the frame B' of the tracking trajectory of the fourth person in the k-1 frame k―1 and speed V′ k―1 , predict the trajectory box position B′ of the fourth person in the kth frame k (second detection frame parameters), and then calculate the current trajectory prediction frame B′ of the fourth person through the intersection and union ratio k and person detection box B k Motion feature similarity D (third detection frame parameter) motion . D motion The calculation of is as follows:

[0092]

[0093] Among them, x′, y′, γ′ and h′ are the trajectory prediction box B′ under the kth frame k The upper left corner horizontal coordinate, upper left corner vertical coordinate, aspect ratio and height; x, y, γ and h are the person detection box B in the kth frame respectively. k The upper left corner horizontal coordinate, upper left corner vertical coordinate, aspect ratio and height. motion The larger the value, the more box B' k With box B kThe closer, the more similar the motion features are.

[0094] In step S320, the appearance feature similarity between the appearance feature parameter of the fourth person in the first tracking trajectory information and the appearance feature parameter of the person in each first person information is determined.

[0095] The device determines the appearance feature F ' k―1 The appearance feature similarity D k―1 between the current trajectory appearance feature F' k and the person appearance feature F apperarnce is calculated by the cosine similarity. apperarnce The calculation of D

[0096]

[0097] wherein ‖F' k―1 ‖ represents the length of the feature vector F' k―1 , and ‖F k ‖ represents the length of the feature vector F k . The greater the value of D appearance is, the more similar the feature vector F' k―1 is to the feature vector F k , that is, the more similar the appearance features are.

[0098] In step S330, a cost matrix is constructed according to each motion feature similarity and each appearance feature similarity, and each first person information and each first tracking trajectory information are matched according to the cost matrix.

[0099] The device calculates the motion feature similarity and the appearance feature similarity between each of the I first person information and each of the J first tracking trajectory information, and generates an I×J size cost matrix, wherein each element D cost is calculated by the following formula:

[0100] D cost = λD appearance + (1-λ)D motion

[0101] wherein λ is an adjustment factor in the interval [0, 1], which is used to adjust the proportion of the two feature similarities. In the embodiment, since the person target occupies a large proportion in the camera screen and is concentrated in the lower left corner, the motion feature of the detection frame before and after is not easy to distinguish, and therefore the appearance feature is more important for distinguishing. After multiple experiments, it is verified that when λ = 0.7, the overall tracking performance of the algorithm is best.

[0102] The device matches the I first personnel information and the J first tracking trajectory information based on the cost matrix, and the first personnel information i and the first tracking trajectory information j matched successfully represent that the personnel correlation between the front frame and the back frame is completed.

[0103] For example, for the first personnel information i and the first tracking trajectory information j, the corresponding element in the cost matrix is D cost，ij If D cost，ij is greater than a set value, the first personnel information i and the first tracking trajectory information j are matched. If the first personnel information i is less than or equal to the set value at each element D cost of the cost matrix, the first personnel information is the second personnel information; if the first tracking trajectory information j is less than or equal to the set value at each element D cost of the cost matrix, the first tracking trajectory information is the third tracking trajectory information.

[0104] In this embodiment, the device constructs a cost matrix based on the first tracking trajectory information and the first personnel information, so as to accurately match each first tracking trajectory information and each first personnel information based on the cost matrix.

[0105] In an embodiment, the first personnel information includes appearance feature parameters of the fourth personnel, and the obtaining of the first personnel information corresponding to each personnel in the second image in the monitoring video includes: obtaining a personnel image of the fourth personnel in the second image according to the second detection box parameter of the fourth personnel in the second image; and inputting the personnel image into an appearance feature extraction model to obtain the appearance feature parameters of the fourth personnel.

[0106] For example, first, the personnel image is cropped based on the detection box of the fourth personnel in the second image, and then the personnel image is input into a pedestrian re-identification model (appearance feature extraction model) for inference, and finally the feature vector of the input full connection layer in the model is extracted as the personnel appearance feature . , where C represents the length of the vector. The appearance feature of the personnel can be the feature on the face, head, or some parts of the person.

[0107] The pedestrian re-identification model is essentially an image classification model. Unlike the conventional classification model which classifies species, the pedestrian re-identification model classifies the pedestrian ID. Through related data training, the model can extract the appearance feature of the pedestrian from the image and accurately classify the pedestrian ID. Therefore, for the same person in the front frame and the back frame, the model can extract similar appearance feature vectors as appearance feature parameters.

[0108] In this embodiment, the personnel image is obtained in the second image, and the appearance feature parameters of the personnel are accurately obtained based on the appearance feature extraction model.

[0109] Referring to Figure 4 , Figure 4 Figure 3 is a flowchart of a method for determining personnel in a yard crane according to an embodiment of the present application. The method comprises the following steps. Figure 2 or Figure 3 Step S110, obtaining a second image and initializing pixel values of the second image to obtain a processed image.

[0110] Step S410, initializing pixel values of the second image to obtain a processed image, and detecting personnel in the second image to obtain a plurality of detection frame images. The detection frame image is a second image containing a detection frame of a person.

[0111] In this embodiment, the device obtains a second image and initializes pixel values of the second image to obtain a processed image. For example, the access control platform and the yard crane ladder area in the second image are labeled using a mask labeling tool, such as the light-colored highlighted area in Figure 5 . The device initializes a single-channel binary mask image with the same size as the second image as the processed image, where H (height) and W (width) represent the length and width of the image.

[0112] The device detects personnel in the second image to obtain a plurality of detection frame images. For example, the personnel in the second image are detected by an instance segmentation model to obtain a plurality of output detection frame images , where I represents the number of personnel detected by the model, and i represents the detection sequence number. The detection frame image is a personnel mask image. Considering detection accuracy and detection speed, a YOLOv8 model can be used as the instance segmentation model.

[0113] Step S420, according to the region where each pixel point in the second image is located, assigning values to each pixel point in the processed image to obtain a first mask image, and according to the detection frame in the detection frame image, assigning values to each pixel point in the detection frame image to obtain a second mask image.

[0114] After obtaining the processed image and the plurality of detection frame images, the device assigns values to each pixel point in the processed image based on the region where each pixel point in the second image is located to obtain a first mask image. For example, the device assigns 0 to each pixel point in the processed image to represent an irrelevant region, and assigns 1 to represent a monitoring region, i.e., according to the labeled monitoring region, assigning values to the pixel value of each pixel point of the processed image , thereby obtaining the first mask image.

[0115] The device values each pixel point of the bounding box image based on the bounding box in the bounding box image to obtain a second mask image. For example, each pixel point in the second mask image M is represented by 0 for the background and 1 for the foreground; B = (x, y, γ, h) represents the bounding box, and the position of the bounding box B in the image is determined by the horizontal coordinate x of the upper left corner, the vertical coordinate y of the upper left corner, the height h, and the aspect ratio γ. It can be understood that each pixel value in the bounding box in the bounding box image is assigned to 1, and the other part of the bounding box image is the background.

[0116] In step S430, the pixel values of the same position pixels in the first mask image and each second mask image are subjected to an XOR operation to obtain the number of target pixels corresponding to the second mask image. The pixel value of the target pixel in the first mask image is different from the pixel value in the second mask image.

[0117] After obtaining the first mask image and each second mask image, the pixel values of the same position pixels in the first mask image and each second mask image are subjected to an XOR operation to obtain the number of target pixels in the second mask image. The pixel value of the target pixel in the first mask image is different from the pixel value in the second mask image. It can be understood that each pixel point in the second mask image is subjected to an XOR operation, so that each pixel point has a corresponding operation result. If the operation result is 1, the pixel value of the pixel point in the first image is different from the pixel value in the second image, and the pixel point is a target pixel.

[0118] In step S440, the second position parameter corresponding to the personnel corresponding to each second mask image is determined according to the ratio between the number of target pixels corresponding to each second mask image and the total number of pixels in the second mask image.

[0119] After determining the number of target pixels, the second position parameter L corresponding to the personnel corresponding to each second mask image is determined based on the ratio between the number of target pixels and the total number of pixels in the second mask image. L can be calculated by the following formula:

[0120]

[0121] Wherein, IoU represents the intersection degree of the first mask image and the second mask image, which is calculated by the ratio between the number of target pixels and the total number of pixels in the second mask image. The larger the IoU is, the larger the intersection area is. represents the XOR operator; Sum(·) represents the statistical operation of the intersection pixels (target pixels); finally, when IoU is greater than 0.1%, it is judged that the current personnel position is on the bridge, and 1 is used to represent, otherwise it is judged to be under the bridge, and 0 is used to represent.

[0122] In the embodiment, the personnel correlation between the front frame and the rear frame can be completed by one matching, which is more efficient than the multiple cascade matching of the conventional tracking algorithm. In addition, in the scene of the gantry door access, the appearance feature is more important than the motion feature in matching. The application of the appearance feature similarity of the personnel appearance feature vector to the cost matrix can effectively reduce the matching error, so that the tracking algorithm has higher robustness and is more suitable for the personnel tracking task in the scene of the gantry door access.

[0123] In an embodiment, the device updates the updated track of the second image. Illustratively, the device acquires the historical times associated with the track identifier corresponding to the first tracking track information with the first personnel information. The historical times refer to the number of matching failures of the first tracking track information without the first personnel information. The device updates the historical times to obtain the target times, that is, the historical times + 1 to obtain the target times. If the target times do not reach the set times, the first tracking track information without the first personnel information is determined as the third tracking track information. If the target times reach the set times, the first tracking track information without the first personnel information is deleted. Illustratively, for the first tracking track information without matching, the number of matching failures is 3 frames without being matched, and the first tracking track information is deleted. The set times are, for example, the above-mentioned 3 times.

[0124] In the embodiment, not only can all tracking track results be accurately analyzed, but also all in-out scene states are covered semantically, so that the method has rationality and interpretability. At the same time, thanks to the mechanism of the method, in actual application, if personnel detection is missed in a frame image, the unmatched track does not affect the global personnel quantity, and will be matched again when detected in the next frame to analyze the state. If a new track is added due to the matching error of the tracking algorithm in a frame, the track does not affect the global personnel quantity, and the error track will be deleted when it is not matched for 3 times. It can be seen that the method has a high fault tolerance rate.

[0125] Reference Figure 6 , Figure 6 Figure 4 is a flowchart of a method for determining personnel in a gantry provided by the present application, which is based on any of the embodiments shown in Figures 2 to 5 Figure 4 is a flowchart of a method for determining personnel in a gantry provided by the present application, which is based on any of the embodiments shown in

[0126] Step S610: acquiring the second speed parameter corresponding to the personnel in each first personnel information.

[0127] Step S620: determining the tracking track information of the personnel in the second image according to the first personnel information and the corresponding second speed parameter.

[0128] In the embodiment, after the first personnel information of each person in the second image is determined, the tracking trajectory information of each person in the second image is set based on the first personnel information.

[0129] For example, when the first personnel information is the second personnel information that is not matched successfully, a second speed parameter V of the person corresponding to the second personnel information is initialized, the second speed parameter V is added to the second personnel information, and the tracking trajectory information of the person in the second image is obtained; when the first personnel information is matched with the first tracking trajectory information, the second speed parameter in the first tracking trajectory information is added to the first personnel information, and the tracking trajectory information of the person corresponding to the first personnel information in the second image is obtained.

[0130] In the embodiment, after the first personnel information and the first tracking trajectory information are matched, the tracking trajectory information of the person in the second image is determined based on the second speed parameter of the person in the first personnel information, so as to prepare for the determination of the number of persons in the next frame image of the gantry crane.

[0131] Corresponding to the method for determining the person in the gantry crane, the application further provides a device for determining the person in the gantry crane. In the following, the device for determining the person in the gantry crane will be described in combination with the method for determining the person in the gantry crane. Figure 7 The device for determining the person in the gantry crane will be described in detail.

[0132] As shown in Figure 7 The device for determining the person in the gantry crane 700 comprises:

[0133] The acquisition module 710 is configured to acquire the first tracking trajectory information of each person in the first image of the monitoring video of the gantry crane, and acquire the first personnel information corresponding to each person in the second image of the monitoring video, the first image being the last frame image of the second image, the first tracking trajectory information comprising a first position parameter of the person in the first image, and the first personnel information comprising a second position parameter of the person in the second image in the gantry crane;

[0134] The matching module 720 is configured to match each first personnel information and each first tracking trajectory information, and obtain the second tracking trajectory information, the third tracking trajectory information and the second personnel information, wherein the second tracking trajectory information is the first tracking trajectory information matched with the first personnel information, the third tracking trajectory information is the first tracking trajectory information not matched with the first personnel information, and the second personnel information is the first personnel information not matched with the first tracking trajectory information;

[0135] The first determination module 730 is configured to determine the first personnel change amount on the gantry crane according to the first position parameter of the first personnel corresponding to the second tracking trajectory information and the second position parameter of the first personnel included in the first personnel information matched by the second tracking trajectory information.

[0136] The second determining module 740 is configured to determine a second personnel change amount on the yard crane according to a second position parameter of a second personnel corresponding to the second personnel information, determine a third personnel change amount on the yard crane according to a first position parameter of a third personnel corresponding to the third tracking trajectory information, and determine a total number of personnel on the yard crane according to the first personnel change amount, the second personnel change amount and the third personnel change amount.

[0137] In an embodiment, the personnel determination apparatus 700 in the yard crane is further configured to:

[0138] predict a second detection frame parameter of the fourth personnel in the second image according to a first detection frame parameter and a first speed parameter of the fourth personnel corresponding to the first tracking trajectory information, and determine a motion feature similarity between the second detection frame parameter and the third detection frame parameter in each first personnel information;

[0139] determine an appearance feature similarity between an appearance feature parameter of the fourth personnel in the first tracking trajectory information and an appearance feature parameter of the personnel in each first personnel information;

[0140] construct a cost matrix according to the various motion feature similarities and the various appearance feature similarities, and match each first personnel information and each first tracking trajectory information according to the cost matrix.

[0141] In an embodiment, the personnel determination apparatus 700 in the yard crane is further configured to:

[0142] obtain a personnel image of the fourth personnel in the second image according to the second detection frame parameter of the fourth personnel in the second image;

[0143] input the personnel image into an appearance feature extraction model to obtain the appearance feature parameter of the fourth personnel.

[0144] In an embodiment, the personnel determination apparatus 700 in the yard crane is further configured to:

[0145] perform pixel value initialization on the second image to obtain a processing image, and perform personnel detection on the second image to obtain a plurality of detection frame images, the detection frame image being a second image containing a detection frame of a personnel;

[0146] value each pixel point in the processing image according to a region where each pixel point in the second image is located to obtain a first mask image, and value each pixel point in the detection frame image according to a detection frame in the detection frame image to obtain a second mask image;

[0147] performing exclusive OR operation on pixel values of the same position pixels in the first mask image and each second mask image to obtain a number of target pixels corresponding to the second mask image, the pixel value of the target pixel in the first mask image being different from the pixel value in the second mask image;

[0148] determining the second position parameter corresponding to the personnel corresponding to each second mask image according to a ratio between the number of target pixels corresponding to each second mask image and a total number of pixels in the second mask image.

[0149] In an embodiment, the device 700 for determining personnel in a yard bridge is further configured to:

[0150] obtain a historical number of times associated with the track identifier corresponding to the first tracking trajectory information that is not matched with the first personnel information, the historical number of times being used to indicate a number of matching failures of the first tracking trajectory information that is not matched with the first personnel information;

[0151] updating the historical number of times to obtain a target number of times;

[0152] when the target number of times does not reach a set number of times, determining the first tracking trajectory information that is not matched with the first personnel information as third tracking trajectory information;

[0153] when the target number of times reaches the set number of times, deleting the first tracking trajectory information that is not matched with the first personnel information.

[0154] In an embodiment, the device 700 for determining personnel in a yard bridge is further configured to:

[0155] comparing the first position parameter and the second position parameter to obtain a first comparison result;

[0156] determining a first personnel quantity correction value corresponding to each first personnel according to the first comparison result of each first personnel, wherein the first comparison result indicates that the first position parameter is equal to the second position parameter, the personnel quantity correction value is 0, the first comparison result indicates that the first position parameter is greater than the second position parameter, the personnel quantity correction value is -1, and the first comparison result indicates that the first position parameter is less than the second position parameter, the personnel quantity correction value is 1;

[0157] determining a first personnel change amount according to the personnel quantity correction value of each first personnel.

[0158] In an embodiment, the device 700 for determining personnel in a yard bridge is further configured to:

[0159] obtaining a second speed parameter corresponding to the personnel in each first personnel information;

[0160] determining tracking trajectory information of the personnel in the second image according to the first personnel information and the corresponding second speed parameter.

[0161] Hereinafter, an electronic device according to an embodiment of the present application will be described with reference to the accompanying drawings. Figure 8

[0162] Figure 8 FIG. 1 illustrates a structural block diagram of an electronic device according to an embodiment of the present application.

[0163] As shown in FIG. 8, the electronic device 800 includes one or more processors 810 and a memory 820. Figure 8 The processor 810 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device 800 to perform desired functions.

[0164] The memory 820 can include one or more computer program products, which can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 810 can execute the program instructions to implement the multivariate expression calculation method of the various embodiments of the present application described above and / or other desired functions.

[0165] In one example, the electronic device 800 can further include an input device 830 and an output device 840, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0166] When the electronic device is a stand-alone device, the input device 830 can be a communication network connector for receiving the acquired input signals from the first device and the second device.

[0167] In addition, the input device 830 can further include, for example, a keyboard, a mouse, and / or the like.

[0168] The output device 840 can output various information, including the determined distance information, direction information, and / or the like, to the outside. The output device 840 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0169] Of course, in order to simplify,

[0170] Figure 8 ​​Only some of the components among the components in the electronic device 800 related to the present application are shown, and components such as a bus, an input / output interface, and the like are omitted. In addition to this, the electronic device 800 can include any other appropriate components according to a specific application.

[0171] The present application also provides a computer readable storage medium, the storage medium storing a computer program, the computer program being used to execute the steps in the method for determining personnel in a yard bridge in the above embodiments.

[0172] In addition to the above method and device, an embodiment of the present application can also be a computer program product, which includes computer program information, and the computer program information causes a processor to execute the steps in the method for determining personnel in a yard bridge according to various embodiments of the present application described in the present specification when the computer program information is run by the processor.

[0173] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as the "C" language or the like. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0174] In addition, an embodiment of the present application can also be a computer readable storage medium, which stores computer program information, and the computer program information causes a processor to execute the steps in the method for determining personnel in a yard bridge according to various embodiments of the present application described in the present specification when the computer program information is run by the processor.

[0175] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above.

[0176] The basic principles of the application are described above in connection with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not intended to be limiting, and it cannot be considered that these advantages, benefits, effects and the like are necessarily possessed by each embodiment of the present application. In addition, the specific details of the above disclosure are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above details do not limit the present application to necessarily adopt the above specific details to realize.

[0177] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, mean "include but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0178] It should also be noted that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

Claims

1. A method for determining personnel in a field crane, characterized in that: include: Obtaining first tracking trajectory information for each person in a first image in a surveillance video of the field bridge, and obtaining first person information corresponding to each person in a second image in the surveillance video, where the first image is a frame previous to the second image, the first tracking trajectory information includes a first position parameter of the person in the first image, and the first person information includes a second position parameter of the person in the second image within the field bridge; Matching each of the first person information and each of the first tracking trajectory information to obtain second tracking trajectory information, third tracking trajectory information, and second person information, wherein the second tracking trajectory information is the first tracking trajectory information matched with the first person information, the third tracking trajectory information is the first tracking trajectory information not matched with the first person information, and the second person information is the first person information not matched with the first tracking trajectory information; determining a change amount of the first person on the field bridge according to a first position parameter of the first person corresponding to the second tracking trajectory information and a second position parameter of the first person included in the first person information matched with the second tracking trajectory information; determining a change amount of the second person on the field bridge according to a second position parameter of the second person corresponding to the second person information, and determining a change amount of the third person on the field bridge according to a first position parameter of the third person corresponding to the third tracking trajectory information; The total number of people on the field crane is determined according to the first personnel change amount, the second personnel change amount, and the third personnel change amount.

2. The method for determining personnel in a field crane according to claim 1, characterized in that: The matching of each piece of first person information and each piece of first tracking trajectory information includes: predicting second detection frame parameters of the fourth person in the second image based on the first detection frame parameters and the first speed parameter of the fourth person corresponding to the first tracking trajectory information, and determining a motion feature similarity between the second detection frame parameters and the third detection frame parameters in each piece of the first person information; determining an appearance feature similarity between an appearance feature parameter of a fourth person in the first tracking trajectory information and an appearance feature parameter of each person in the first person information; A cost matrix is ​​constructed according to the similarities of the motion features and the similarities of the appearance features, and the first person information and the first tracking trajectory information are matched according to the cost matrix.

3. The method for determining personnel in a field crane according to claim 2, characterized in that: The first person information includes appearance feature parameters of a fourth person, and obtaining the first person information corresponding to each person in the second image in the surveillance video includes: Acquire a person image of the fourth person in the second image according to a second detection frame parameter of the fourth person in the second image; The person image is input into an appearance feature extraction model to obtain appearance feature parameters of the fourth person.

4. The method for determining personnel in a field crane according to claim 1, characterized in that: The obtaining of first person information corresponding to each person in the second image in the surveillance video includes: Initializing pixel values ​​of the second image to obtain a processed image, and detecting people in the second image to obtain a plurality of detection frame images, each of the detection frame images being the second image containing a detection frame of a person; Assigning a value to each pixel in the processed image according to the region where each pixel in the second image is located to obtain a first mask image, and assigning a value to each pixel in the detection frame image according to the detection frame in the detection frame image to obtain a second mask image; performing an exclusive-OR operation on pixel values ​​at the same position in the first mask image and each of the second mask images to obtain the number of target pixels corresponding to the second mask image, where the pixel value of the target pixel in the first mask image is different from the pixel value in the second mask image; The second position parameter corresponding to the person corresponding to each second mask image is determined according to a ratio between the number of target pixels corresponding to each second mask image and the total number of pixels in the second mask image.

5. The method for determining personnel in a field crane according to claim 1, characterized in that: Also includes: Obtaining a historical number of times associated with a trajectory identifier corresponding to first tracking trajectory information that is not matched with first person information, where the historical number is used to indicate a number of matching failures corresponding to the first tracking trajectory information that is not matched with first person information; Updating the historical number of times to obtain a target number of times; If the target number of times does not reach the set number of times, the first tracking trajectory information that is not matched with the first person information is determined as the third tracking trajectory information; When the target number of times reaches the set number of times, the first tracking trajectory information that is not matched with the first person information is deleted.

6. The method for determining personnel in a field crane according to claim 1, characterized in that: The determining, based on the first position parameter of the first person corresponding to the second tracking trajectory information and the second position parameter of the first person included in the first person information matched by the second tracking trajectory information, of a first person change on the field bridge includes: comparing a first comparison result of the first position parameter and the second position parameter; determining, based on a first comparison result for each of the first persons, a first person quantity correction value corresponding to each of the first persons, wherein if the first comparison result indicates that the first position parameter is equal to the second position parameter, the person quantity correction value is 0; if the first comparison result indicates that the first position parameter is greater than the second position parameter, the person quantity correction value is -1; and if the first comparison result indicates that the first position parameter is less than the second position parameter, the person quantity correction value is 1; The first personnel change amount is determined according to the personnel quantity correction value of each of the first personnel.

7. The method for determining personnel in a field crane according to any one of claims 1 to 6, characterized in that: After matching each of the first personnel information and each of the first tracking trajectory information, the method further includes: Obtaining a second speed parameter corresponding to each person in the first person information; According to the first person information and the corresponding second speed parameter, tracking trajectory information of the person in the second image is determined.

8. A device for identifying people in a field crane, characterized in that: include: an acquisition module, configured to acquire first tracking trajectory information of each person in a first image in a surveillance video of the field bridge, and acquire first person information corresponding to each person in a second image in the surveillance video, wherein the first image is a frame previous to the second image, the first tracking trajectory information includes first position parameters of the person in the first image, and the first person information includes second position parameters of the person in the second image within the field bridge; a matching module, configured to match each piece of the first person information with each piece of the first tracking trajectory information to obtain second tracking trajectory information, third tracking trajectory information, and second person information, wherein the second tracking trajectory information is the first tracking trajectory information matched with the first person information, the third tracking trajectory information is the first tracking trajectory information not matched with the first person information, and the second person information is the first person information not matched with the first tracking trajectory information; a first determining module, configured to determine a change amount of the first person on the field bridge according to a first position parameter of the first person corresponding to the second tracking trajectory information and a second position parameter of the first person included in the first person information matched with the second tracking trajectory information; The second determination module is used to determine the change in the second person on the field bridge based on the second position parameter of the second person corresponding to the second person information, and to determine the change in the third person on the field bridge based on the first position parameter of the third person corresponding to the third tracking trajectory information, and to determine the total number of people on the field bridge based on the first person change, the second person change and the third person change.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein, The memory is connected to the processor and is used to store programs; The processor is configured to implement the method for determining a person in a field bridge according to any one of claims 1 to 7 by running the program in the memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining a person in a field bridge according to any one of claims 1 to 7 is implemented.

Images