Visual identification method, visual identification device, storage medium and equipment

Abstract

The invention discloses a visual identification method, a visual identification device, a storage medium and equipment, and relates to the technical field of pipe piles. The method comprises the following steps: calibrating an ideal identification position of each station, and configuring a floating coefficient to obtain calibration data; decoding a video frame collected by a camera into a picture, and inputting the picture into a neural network to obtain target identification data; and judging whether the target enters a station or leaves the station based on the calibration data and the target identification data. According to the invention, the ideal identification position of each station is calibrated, the video frame collected by the camera is decoded into the picture to be input into the neural network, whether the target enters the station or leaves the station is judged in sequence, compared with an RFID chip, the manufacturing cost is low, the durability is realized, the maintenance cost and the hardware cost are reduced, and the working efficiency is improved. And the device can adapt to various severe environments.

Description

Technical Field

[0001] The present invention relates to the technical field of pipe piles, and in particular to a visual recognition method, a visual recognition device, a storage medium and equipment. Background Art

[0002] In the production process of the pipe pile industry, it is necessary to manage the information of the flow of semi-finished products. In the traditional manual production scenario, this can only be achieved by manually registering the mold number identification code carried by the mold in the key process and combining it with the temporary specification information. The current mainstream mold number automatic identification method is to use RFID and then install a card reader at the workstation. Compared with the traditional manual production scenario, this method greatly improves the recognition accuracy and efficiency. However, this method requires a large number of RFID chips and card reader equipment. The vibration of the RFID chip during centrifugal production and the high temperature and high pressure during steaming and autoclaving result in a high loss rate of the RFID chip. The high maintenance frequency leads to high maintenance costs and data collection costs. Damage to the RFID chip will also affect the recognition rate. Summary of the invention

[0003] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a visual recognition method, a visual recognition device, a storage medium and a device for solving the problems brought about by the existing RFID chip solutions.

[0004] According to a first aspect of the present invention, there is provided a visual recognition method, comprising:

[0005] Calibrate the ideal identification position of each workstation and configure the floating coefficient to obtain calibration data;

[0006] Decode the video frames captured by the camera into images and input them into the neural network to obtain target recognition data;

[0007] Based on the calibration data and the target recognition data, it is determined whether the target enters or leaves the workstation.

[0008] The visual recognition method of the present invention calibrates the ideal recognition position of each workstation, and then decodes the video frames captured by the camera into pictures and inputs them into the neural network to judge whether the target enters or leaves the workstation. Compared with RFID chips, the visual recognition method of the present invention is low-cost and durable, reduces maintenance costs and hardware costs, and can adapt to a variety of harsh environments.

[0009] In some implementations, the step of calibrating the ideal identification position of each workstation and configuring a floating coefficient to obtain calibration data includes:

[0010] The ideal identification position of each station is calibrated with station n as the unit, and the floating coefficient is configured to obtain the calibration data set BDZB n:{workid n , xyvalf n ,areavalf n , xval n ,yval n , wval n , hval n};

[0011] Among them, workid n is the ID of station n, xyvalf n To identify the range floating coefficient, areavalf n xval is the area floating coefficient of the mold base on the identified target mold, n The x value of the upper left corner coordinate of the identification box n, yval n is the y value of the upper left corner coordinate of the identification box n, wval n hval is the width of the recognition box. n is the height of the recognition box,

[0012] In some implementations, decoding the video frames captured by the camera into images and inputting them into a neural network to obtain target recognition data includes:

[0013] The image decoded from the video frame collected by the camera is recognized by the neural network to obtain the target recognition data, wherein the target recognition data includes the current recognition result and the set of coordinates SBZB m :{X m , Y m , W m , H m , T m , A.C. m};

[0014] Among them, X m is the X value of the coordinate of the point in the actual recognition frame, Y m is the Y value of the coordinates of the point in the actual recognition frame, W m is the actual width of the recognition box, H m is the height of the actual recognition box, T m is the label on the target, AC m To identify the similarity,

[0015] In some embodiments, the determining whether the target enters or leaves the workstation based on the calibration data and the target recognition data includes:

[0016] Screening the target recognition data based on the recognition similarity to obtain a screening result;

[0017] Matching the acquired current workstation with the calibration data to obtain a matching result;

[0018] Based on the matching result and the screening result, the label on the target is judged to obtain a label judgment result;

[0019] Obtaining coordinates corresponding to the target recognition data based on the label judgment result;

[0020] Based on the coordinates corresponding to the target identification data, the inclusion relationship of each part of the target is determined to obtain an inclusion relationship result, wherein each part of the target includes a mold, a mold base plate set on the mold, and a base plate number on the mold base plate;

[0021] Based on the inclusion relationship result, determine whether the relative position of the identification point on the mold base plate is correct, and obtain the identification point determination result;

[0022] Based on the identification point judgment result, it is judged whether the area of ​​the mold bottom plate meets the preset conditions, and the area judgment result of the mold bottom plate is obtained;

[0023] Based on the area judgment result of the mold bottom plate and the number of digits in the inclusion relationship result judgment module bottom plate, a number quantity judgment result is obtained;

[0024] Based on the result of the number determination, the numbers on the mold base plate are sorted to obtain a final mold number;

[0025] It is determined whether the target enters or leaves the workstation based on the final model number.

[0026] In some implementations, the screening of the target recognition data based on the recognition similarity to obtain a screening result includes:

[0027] Traversing the collection SBZB m , according to the recognition similarity AC m Compare the preset specified similarity values ​​to obtain the actual recognition similarity rate XS m , the actual recognition similarity rate XS m The calculation formula is:

[0028] XS m =AC m ×100% - specifies the similarity value;

[0029] Among them, when XS m ≥0, keep SBZB m When XS m <0, discard the set SBZB m ;

[0030] The step of matching the acquired current workstation with the calibration data to obtain a matching result includes:

[0031] Traversing the collection BDZB n , and obtain the current workstation Aworkid and the collection BDZB n workid n Match, if the match result is a match, the output result is BDZB m ;

[0032] The step of judging the label on the target based on the matching result and the screening result to obtain the label judgment result includes:

[0033] Traversing the collection SBZB m , according to T m Determine whether there is a mold label, a mold base plate label, or a digital label. If the result is yes, save SBZB m Subscript value in set XB i , where 0≤i<14;

[0034] The obtaining of coordinates corresponding to the target recognition data based on the tag judgment result includes:

[0035] Traverse the collection XB ij The data in the calculation set Coordinates and form a set The calculation formula includes:

[0036]

[0037]

[0038]

[0039]

[0040] Formula calculation to get the set

[0041]

[0042] Among them, j∈m;

[0043] The step of determining the inclusion relationship of each part of the target based on the coordinates corresponding to the target identification data to obtain the inclusion relationship result includes:

[0044] Traversal Perform the limited condition judgment: whether the mold base is completely contained by the mold, whether the base number is completely contained by the mold base, where the judgment subject is d e, the outer layer subject is st, then the judgment formula is:

[0045] LX0 de -LX0 st ≥0

[0046] LX0 de -LX2 st ≤0

[0047] LY0 de -LY0 st ≥0

[0048] LY0 de -LY2 st ≤0

[0049] LX2 de -LX0 st ≥0;

[0050] LX2 de -LX2 st ≤0

[0051] LY2 de -LY0 st ≥0

[0052] LY2 de -LY2 st ≤0

[0053] When the above inequalities are true at the same time, the record base number is completely contained in the number set of the module base

[0054] The step of judging whether the area of ​​the mold bottom plate meets a preset condition based on the identification point judgment result to obtain the area judgment result of the mold bottom plate includes:

[0055] According to the collection Combined with the set BDZB, calculate whether the area of ​​the mold base meets the preset conditions The calculation formula is:

[0056]

[0057] when When , it means that the preset conditions are met. , it means that the preset conditions are not met;

[0058] The method of determining the number of digits in the module bottom plate based on the area determination result of the mold bottom plate and the inclusion relationship result to obtain the number determination result includes:

[0059] when When the record is MBN K:{M k , B k , N1 k , N2 k , N3 k , N4 k},

[0060] Among them, M is the mold in the set SBZB m The subscript in the SBZB is the mold base plate. m The subscript in N1 is the number 1 of the mold base in the set SBZB m The subscript in N2 is the number 2 of the mold base in the set SBZB m The subscript in N3 is the number 3 of the mold base in the set SBZB m The subscript in N4 is the number 4 of the mold base in the set SBZB m The subscript in

[0061] The method of sorting the numbers on the mold base plate based on the number determination result to obtain the final mold number includes:

[0062] Traversing the MBN collection K , will collect SBZB m In Sort in ascending order and put Combined into digital output CODE.

[0063] In some embodiments, judging whether the target enters or leaves the workstation based on the final model number includes:

[0064] When a valid CODE is identified, it is recorded as the mold entering the station;

[0065] When the identified CODE disappears or is updated, it is recorded as the mold leaving the station.

[0066] In some embodiments, it further comprises:

[0067] When the mold is recorded as entering the workstation, the entry time is recorded;

[0068] When it is recorded that the mold leaves the workstation, the departure time is recorded;

[0069] According to the recorded entry time and the recorded exit time, the residence time of the mold in the workstation is obtained.

[0070] According to a second aspect of the present invention, a visual recognition device is provided, the visual recognition device is used to perform the above-mentioned visual recognition method, comprising:

[0071] The calibration module is used to calibrate the ideal identification position of each station and configure the floating coefficient to obtain the calibration data;

[0072] The recognition module is used to decode the video frames collected by the camera into pictures and input them into the neural network to obtain target recognition data;

[0073] A judgment module is used to judge whether the target enters or leaves the workstation based on the calibration data and the target recognition data.

[0074] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0075] at least one processor; and

[0076] a memory communicatively connected to the at least one processor; wherein,

[0077] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned visual recognition method.

[0078] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the above-mentioned visual recognition method when executed by a processor.

[0079] Compared with the prior art, the visual recognition method, visual recognition device, storage medium and equipment of the present invention calibrate the ideal recognition position of each workstation, and then decode the video frames captured by the camera into pictures and input them into the neural network to judge whether the target enters or leaves the workstation in turn. Compared with RFID chips, they are low-cost and durable, reduce maintenance costs and hardware costs, and can adapt to a variety of harsh environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a structural schematic diagram of a mold base plate according to an embodiment of the present invention;

[0081] Figure 2 is a flow chart of a visual recognition method according to one embodiment of the present invention;

[0082] Figure 3 A schematic diagram of coordinates of an identification frame according to an embodiment of the present invention;

[0083] Figure 4 A flowchart for determining whether a mold has entered a work station according to an embodiment of the present invention;

[0084] Figure 5 A flowchart of judging whether a mold has entered a workstation based on a mold number according to an embodiment of the present invention;

[0085] Figure 6 A schematic diagram of the structure of a visual recognition device according to an embodiment of the present invention;

[0086] Figure 7 A schematic diagram of data flow in a visual recognition process according to an embodiment of the present invention;

[0087] Figure 8 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0088] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0089] The current commonly used solution is RFID chip + card reader. This solution is to build the chip into the mold and install a card reader at each workstation to identify the RFID chip that appears in the identification range. The hardware cost of this method is high, and the chip is easily damaged in a high temperature, high pressure, high humidity and strong vibration environment, which leads to increased maintenance costs and increased continuous hardware costs.

[0090] To this end, the present invention solves the above problems by collecting video with a camera + AI recognition + algorithm, wherein the mold number is an inherent mark on the mold, and a mold number base plate 10 is provided on the mold, such as Figure 1 As shown, there are three identification points 11 and a mold base plate number 12 on the mold base plate 10. The identification point 11 can help determine the location of the occlusion angle and can be used to exclude dirty data caused by similar boards. The algorithm will determine that the mold base plate that meets the characteristics of the mold base plate is a valid mold base plate. In order to meet the needs of identification, overcome the harsh conditions of workshop production, and reduce maintenance costs, the specific numbers in the mold base plate number 12 can be replaced as needed. The font of the mold base plate number 12 is a hollow font with greater differences, and the width of the connecting ribs is widened. The identification point 11 will not increase the cost of the original mold base plate. The specifications and models used by the camera are also ordinary industrial cameras, which are low in cost and durable, reduce maintenance costs and hardware costs, and can adapt to a variety of harsh environments.

[0091] The embodiment of the present invention provides a visual recognition method, such as Figure 2 As shown, the visual recognition method includes:

[0092] S100: Calibrate the ideal identification position of each workstation and configure the floating coefficient to obtain calibration data;

[0093] In this embodiment, the workstations are numbered so that each workstation obtains a unique workstation ID, the ideal identification position of each workstation is calibrated, and a floating coefficient is configured to determine the ideal identification frame. During formal identification, the data will be identified within the error of the floating coefficient. If the object to be identified is not in the identification area, it is considered to be invalid data.

[0094] In an optional implementation, step S100 calibrates the ideal identification position of each workstation and configures a floating coefficient to obtain calibration data, including:

[0095] The ideal identification position of each station is calibrated with station n as the unit, and the floating coefficient is configured to obtain the calibration data set BDZB n , collection BDZB n for:

[0096] BDZB1: {workid1, xyvalf1, areavalf1, xval1, yval1, wval1, hval1}

[0097] BDZB2: {workid2,xyvalf2,areavalf2,xval2,yval2,wval2,hval2}

[0098] …

[0099] BD n :{workid n , xyvalf n ,areavalf n , xval n ,yval n , wval n , hval n};

[0100] Among them, workid n is the ID of workstation n, in the set BDZB n is unique in xyvalf n To identify the range floating coefficient, areavalf n xval is the area floating coefficient of the mold base on the identified target mold, n The x value of the upper left corner coordinate of the identification box n, yval n is the y value of the upper left corner coordinate of the identification box n, wval n hval is the width of the recognition box. n is the height of the recognition box,

[0101] S200: Decode the video frames collected by the camera into images and input them into the neural network to obtain target recognition data:

[0102] In this embodiment, AI (such as OPENCV network neural network) is trained through artificial neural network to recognize the pictures decoded from the video frames collected and returned by the camera, and the mold, mold base or mold base number in the picture is extracted. After the mold, mold base or mold base number is identified, subsequent object similarity, object recognition coordinates, object recognition frame width and height and other data can be used for subsequent legality judgment.

[0103] In an optional implementation, step S200 decodes the video frames captured by the camera into images and inputs them into a neural network to obtain target recognition data, including:

[0104] The neural network is used to identify the images decoded from the video frames collected by the camera to obtain target recognition data, where the target recognition data includes the current recognition result and the set of coordinates SBZB m , collection SBZB m for:

[0105] SBZB1: {X1, Y1, W1, H1, T1, AC1}

[0106] SBZB2: {X2, Y2, W2, H2, T2, AC2}

[0107] …

[0108] BYZGR m :{X m , Y m , W m , H m , T m , A.C. m};

[0109] Among them, Figure 3 As shown, X m is the X value of the coordinates of the actual point in the recognition frame (i.e. Figure 3 X4), Y m is the Y value of the coordinates of the actual point in the recognition frame (i.e. Figure 3 Y4), W m is the actual width of the recognition box, H m is the height of the actual recognition box, T m is the label on the target, AC m To identify the similarity,

[0110] S300: Determine whether the target enters or leaves the workstation based on the calibration data and the target recognition data.

[0111] In an optional embodiment, if Figure 4 As shown, step S300 determines whether the target enters or leaves the workstation based on the calibration data and the target recognition data, including:

[0112] S310: Screening the target recognition data based on the recognition similarity to obtain a screening result;

[0113] In this embodiment, the target recognition data is screened based on the recognition similarity to obtain the screening result, including the screening function fxs(SBZB m ) Get the screening results, specifically, traverse the set SBZB m , according to the recognition similarity AC m Compare the preset specified similarity value (the corresponding value can be set as needed) to obtain the actual recognition similarity rate XS m , the actual recognition similarity rate XS m The calculation formula is:

[0114] XS m =AC m ×100% - specifies similarity value:

[0115] Among them, when XS m ≥0, keep SBZB m When XS m <0, discard the set SBZB m , and the discarded data no longer participates in subsequent calculations.

[0116] S320: Match the acquired current workstation with the calibration data to obtain a matching result;

[0117] In this embodiment, the current workstation obtained is matched with the calibration data to obtain a matching result, including obtaining a matching result through a matching function fpp(Aworkid). Specifically, the set BDZB is traversed. n , and obtain the current workstation Aworkid and the collection BDZB n workid n Match, if the match result is a match (i.e. legal), the output result is BDZB m , otherwise it is deemed illegal and will no longer participate in subsequent calculations.

[0118] S330: judging the label on the target based on the matching result and the screening result to obtain a label judgment result;

[0119] In this embodiment, the label on the target is judged based on the matching result and the screening result to obtain the label judgment result, including when the fpp (Aworkid) result is legal (i.e., matching), performing the label judgment function flxpd (SBZB m ), specifically, traverse the set SBZB m , according to T m Determine whether there is a mold label, a mold base plate label, or a digital label. If the result is yes, save SBZB m Subscript value in set XB i Otherwise, the judgment result is abnormal and an abnormal mark needs to be returned; among them, 0≤i<14; the i value distribution rule is: the upper mold corresponds to number 0, the lower mold corresponds to number 1, the whole mold corresponds to number 2, the mold base corresponds to number 3, and 0~9 correspond to numbers 4~13.

[0120] S340: Obtaining coordinates corresponding to the target recognition data based on the label determination result;

[0121] In this embodiment, the coordinates corresponding to the target identification data are obtained based on the tag judgment result, including when XB i When the data with the model base plate in the label exists, the coordinates of the data are calculated and output as a set in combination with BDZB The calculation function is fwzpd(SBZB m ), specifically, traverse the set XB ij The data in the calculation set Coordinates and form a set The calculation formula includes:

[0122]

[0123]

[0124]

[0125]

[0126] Formula calculation to get the set gather for:

[0127] ALLZB1: {LX01, LY01, LX11, LY11, LX21, LY21, LX31, LY31}

[0128] ALLZB2: {LX02, LY02, LX12, LY12, LX22, LY22, LX32, LY32}

[0129] …

[0130] Among them, j∈m, the set The coordinates in correspond to Figure 3 X0, Y0, X1, Y1, X2, Y2, X3, Y3 shown.

[0131] S350: judging the inclusion relationship of each part of the target based on the coordinates corresponding to the target recognition data, and obtaining an inclusion relationship result, wherein each part of the target includes a mold, a mold number base plate set on the mold, and a base plate number on the mold number base plate;

[0132] In this embodiment, based on the coordinates corresponding to the target identification data, the inclusion relationship of each part of the target is determined to obtain the inclusion relationship result, including: To judge the legitimacy of the position, the judgment function is Specifically, traverse The limiting conditions are judged: whether the mold base is completely contained by the mold, whether the base number is completely contained by the mold base, where the judging subject is de and the outer layer is judged by st, then the judgment formula is:

[0133] LX0 de -LX0 st ≥0

[0134] LX0 de -LX2 st ≤0

[0135] LY0 de -LY0 st ≥0

[0136] LY0 de -LY2 st ≤0

[0137] LX2 de -LX0 st ≥0;

[0138] LX2 de -LX2 st ≤0

[0139] LY2 de -LY0 st ≥0

[0140] LY2 de -LY2 st ≤0

[0141] When the above inequalities are true at the same time, it means that the limit condition is passed and the number of base plate numbers that are completely contained in the module base plate is recorded. Otherwise, it does not pass the limited condition judgment and no longer participates in subsequent calculations. This step can determine whether there is a situation where the model base plate is manually held. This algorithm can improve the accuracy and recognition stability.

[0142] S360: judging whether the relative position of the identification point on the mold base plate is correct based on the inclusion relationship result, and obtaining the identification point judgment result;

[0143] In this embodiment, The function will first be used to determine whether the mold base is completely contained by the mold. When the result is passed, it will then determine whether the three identification points of the mold base exist and their relative positions are correct. When the result is passed, it will enter the next step of calculation.

[0144] S370: judging whether the area of ​​the mold bottom plate meets a preset condition based on the identification point judgment result, and obtaining the area judgment result of the mold bottom plate;

[0145] In this embodiment, the area of ​​the mold bottom plate is judged based on the identification point judgment result to determine whether it meets the preset conditions, and the area judgment result of the mold bottom plate is obtained, including when judging whether the three identification points of the mold bottom plate exist and the relative positions are correct, the area of ​​the mold bottom plate is judged based on the set. Combined with the set BDZB, the area legality of the mold base is judged, and the judgment function is: Specifically, according to the collection Combined with the set BDZB, calculate whether the area of ​​the mold base meets the preset conditions The calculation formula is:

[0146]

[0147] when When , it means that the preset conditions are met, that is, the area is judged to be legal. , it means that the preset conditions are not met, that is, the area does not pass the legal judgment, and the base plate of this model number cannot participate in subsequent calculations.

[0148] S380: judging the number of digits in the module bottom plate based on the area judgment result of the mold bottom plate and the inclusion relationship result, and obtaining a number judgment result;

[0149] In this embodiment, based on the area judgment result of the mold bottom plate and the number of digits in the inclusion relationship result judgment module bottom plate, the number judgment result is obtained, including the mold bottom plate that passes the area legal judgment to continue the subsequent calculation, according to the set Determine the legitimacy of the number of numbers in the module baseplate. The determination method is:

[0150] when (If the preset value is 4), it is an abnormal situation and the abnormal value needs to be recorded;

[0151] when (If the default value is 4), the legitimacy of the number is determined and recorded as a set MBN K :

[0152] when (If the preset value is 4), it is an abnormal situation and the abnormal value needs to be recorded;

[0153] MBN Collection K for:

[0154] MBN1: {M1, B1, N11, N21, N31, N41}

[0155] MBN2: {M2, B2, N12, N22, N32, N42}

[0156] …

[0157] MBN K :{M k , B k , N1 k , N2 k , N3 k , N4 k};

[0158] Among them, M is the mold in the set SBZB m The subscript in the SBZB is the mold base plate. m The subscript in N1 is the number 1 of the mold base in the set SBZB m The subscript in N2 is the number 2 of the mold base in the set SBZB m The subscript in N3 is the number 3 of the mold base in the set SBZB m The subscript in N4 is the number 4 of the mold base in the set SBZB m The subscript in

[0159] S390: sorting the numbers on the mold base plate based on the number determination result to obtain a final mold number;

[0160] In this embodiment, the numbers on the mold base are sorted based on the number determination result to obtain the final mold number, including the content set MBN output according to the result of the data legitimacy determination. K , for each mold base plate, the digital sorting of the mold base plate is performed, and the digital sorting function is fpx(MBN K ), so as to obtain the final module number CODE, specifically, traverse the set MBN K , will collect SBZB m In Sort in ascending order and put Combined into digital output CODE.

[0161] S391: Determine whether the target enters or leaves the workstation based on the final model number.

[0162] In an optional embodiment, if Figure 5 As shown, step S391 determines whether the target enters or leaves the workstation based on the final model number, including:

[0163] S391A: When a valid CODE is identified, it is recorded as the mold entering the station;

[0164] In this embodiment, the occurrence node of CODE is detected, and when a valid CODE is identified, it is recorded as the mold entering the station, and the entry time is recorded.

[0165] S391B: When the recognized CODE disappears or is updated, it is recorded as the mold leaving the station.

[0166] In this embodiment, the disappearing nodes of CODE are continuously detected. When the disappearance or update of CODE is identified, it is recorded as the mold leaving the station, and the leaving time is recorded.

[0167] Furthermore, the residence time of the mold at the workstation is obtained based on the recorded entry time and the recorded exit time, that is, the difference between the recorded entry time and the exit time of the mold is the residence time of the mold at the workstation.

[0168] The implementation method of the present invention calibrates the ideal identification position of each workstation, and then decodes the video frames captured by the camera into images and inputs them into the neural network to determine whether the target enters or leaves the workstation. Compared with RFID chips, it is low-cost and durable, reduces maintenance costs and hardware costs, and can adapt to a variety of harsh environments.

[0169] The embodiment of the present invention provides a visual recognition device, such as Figure 6 As shown, the device can be used as a specific device to implement the visual recognition method provided by the embodiment of the present invention. The device may include:

[0170] The calibration module 100 is used to calibrate the ideal identification position of each workstation and configure the floating coefficient to obtain calibration data;

[0171] The recognition module 200 is used to decode the video frames collected by the camera into pictures and input them into the neural network to obtain target recognition data;

[0172] The judgment module 300 is used to judge whether the target enters or leaves the workstation based on the calibration data and the target recognition data.

[0173] It should be noted that the steps involved in the visual recognition device have been described in detail in the visual recognition method, so they will not be repeated here.

[0174] In an optional embodiment, if Figure 7 As shown, the data flow during the visual recognition process of the visual recognition device includes:

[0175] A11: Workstation calibration, that is, calibrate each workstation;

[0176] A12: basic data of the workstation, i.e., the ID of workstation n, the floating coefficient of the identification range, and the area floating coefficient of the mold base plate on the identified target mold;

[0177] A13: Get calibration data based on station calibration and station basic data;

[0178] A14: Convert calibration data into basic data blocks;

[0179] B11: Camera acquisition, that is, the camera acquires the captured video from the workstation;

[0180] B12: Obtaining a video frame based on the collected video;

[0181] B13: Input the video frame into the neural network recognition module, that is, recognize the image decoded from the video frame;

[0182] C11: Obtain recognition results and coordinates, that is, obtain a set of recognition results and coordinates;

[0183] C12: Determine whether the mold enters the workstation, i.e., determine whether the target enters the workstation through calibration data and target recognition data;

[0184] C13: The data legitimacy judgment module judges the status of the mold and the workstation, that is, the relationship between the mold and the workstation is judged by the data legitimacy judgment module, such as entering, staying or leaving;

[0185] C14: Get the mold number data block, that is, get the CODE. Once the CODE is obtained, it is recorded as the mold entering the workstation;

[0186] C15: The station mold leaving detection module detects whether the mold has left the station, that is, when the recognized CODE disappears or is updated, it is recorded as the mold leaving the station;

[0187] C16: The module for calculating the mold residence time calculates the residence time of the mold at the workstation, that is, the difference between the recorded mold entry time and exit time is the residence time of the mold at the workstation;

[0188] D11: The communication module receives data on the mold entering the workstation, the mold leaving the workstation, and the dwell time at the workstation, and sends it to the host computer.

[0189] An embodiment of the present invention provides an electronic device, such as Figure 8 As shown, Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0190] like Figure 8 As shown, the electronic device is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors 910, a memory 930, and a communication bus 940 connecting different system components (including the memory 930 and the processor 910).

[0191] The communication bus 940 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnection (PCI) bus.

[0192] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0193] The memory 930 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Figure 8Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 940 via one or more data medium interfaces. The memory 930 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0194] A program / utility having a set (at least one) of program modules may be stored in memory 930, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.

[0195] The electronic device may also communicate with one or more external devices, and may also communicate with one or more devices that enable a user to interact with the electronic device, or communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through the communication interface 920. In addition, the electronic device may also communicate with the network adapter ( Figure 8 The network adapter can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the communication bus 940. It should be understood that although Figure 8 Other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Drives; hereafter referred to as: RAID) systems, tape drives, and data backup storage systems, etc.

[0196] The processor 910 executes various functional applications and data processing by running the programs stored in the memory 930, such as implementing the visual recognition method provided in the embodiment of the present invention.

[0197] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the visual recognition method provided by the embodiment of the present invention.

[0198] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above, but is not limited to this. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0199] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0200] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0201] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A visual recognition method, characterized in that: include: Calibrate the ideal identification position of each workstation and configure the floating coefficient to obtain calibration data; Decode the video frames captured by the camera into images and input them into the neural network to obtain target recognition data; Based on the calibration data and the target recognition data, it is determined whether the target enters or leaves the workstation.

2. The visual recognition method according to claim 1, characterized in that: The ideal identification position of each workstation is calibrated, and a floating coefficient is configured to obtain calibration data, including: The ideal identification position of each station is calibrated with station n as the unit, and the floating coefficient is configured to obtain the calibration data set BDZB n :{workid n ,xyvalf n ,areavalf n ,xval n ,yval n ,wval n ,hval n }; Among them, workid n is the ID of station n, xyvalf n To identify the range floating coefficient, areavalf n xval is the area floating coefficient of the mold base on the identified target mold, n The x value of the upper left corner coordinate of the identification box n, yval n is the y value of the upper left corner coordinate of the identification box n, wval n hval is the width of the recognition box. n is the height of the recognition box, 3. The visual recognition method according to claim 2, characterized in that: The video frames captured by the camera are decoded into pictures and input into the neural network to obtain target recognition data, including: The image decoded from the video frame collected by the camera is recognized by the neural network to obtain the target recognition data, wherein the target recognition data includes the current recognition result and the set of coordinates SBZB m :{X m ,Y m ,W m ,H m ,T m ,AC m }; Among them, X m is the X value of the coordinate of the point in the actual recognition frame, Y m is the Y value of the coordinates of the point in the actual recognition frame, W m is the actual width of the recognition box, H m is the height of the actual recognition box, T m is the label on the target, AC m To identify the similarity, 4. The visual recognition method according to claim 3, characterized in that: The determining whether the target enters or leaves the workstation based on the calibration data and the target recognition data includes: Screening the target recognition data based on the recognition similarity to obtain a screening result; Matching the acquired current workstation with the calibration data to obtain a matching result; Based on the matching result and the screening result, the label on the target is judged to obtain a label judgment result; Obtaining coordinates corresponding to the target recognition data based on the label judgment result; Based on the coordinates corresponding to the target identification data, the inclusion relationship of each part of the target is determined to obtain an inclusion relationship result, wherein each part of the target includes a mold, a mold base plate set on the mold, and a base plate number on the mold base plate; Based on the inclusion relationship result, determine whether the relative position of the identification point on the mold base plate is correct, and obtain the identification point determination result; Based on the identification point judgment result, it is judged whether the area of ​​the mold bottom plate meets the preset conditions, and the area judgment result of the mold bottom plate is obtained; Based on the area judgment result of the mold bottom plate and the number of digits in the inclusion relationship result judgment module bottom plate, a number quantity judgment result is obtained; Based on the result of the number determination, the numbers on the mold base plate are sorted to obtain a final mold number; It is determined whether the target enters or leaves the workstation based on the final model number.

5. The visual recognition method according to claim 4, characterized in that: The target recognition data is screened based on the recognition similarity to obtain a screening result, including: Traversing the collection SBZB m , according to the recognition similarity AC m Compare the preset specified similarity values ​​to obtain the actual recognition similarity rate XS m , the actual recognition similarity rate XS m The calculation formula is: XS m =AC m ×100% - specifies the similarity value; Among them, when XS m ≥0, keep SBZB m When XS m <0, discard the set SBZB m ; The step of matching the acquired current workstation with the calibration data to obtain a matching result includes: Traversing the collection BDZB n , and obtain the current workstation Aworkid and the collection BDZB n workid n Match, if the match result is a match, the output result is BDZB m ; The step of judging the label on the target based on the matching result and the screening result to obtain the label judgment result includes: Traversing the collection SBZB m , according to T m Determine whether there is a mold label, a mold base plate label, or a digital label. If the result is yes, save SBZB m Subscript value in set XB i , where 0≤i<14; The obtaining of coordinates corresponding to the target recognition data based on the tag judgment result includes: Traverse the collection XB ij The data in the calculation set Coordinates and form a set The calculation formula includes: Formula calculation to get the set Among them, j∈m; The step of determining the inclusion relationship of each part of the target based on the coordinates corresponding to the target identification data to obtain the inclusion relationship result includes: Traversal The limiting conditions are judged: whether the mold base is completely contained by the mold, whether the base number is completely contained by the mold base, where the judging subject is de and the outer layer is judged by st, then the judgment formula is: LX0 de -LX0 st ≥0 LX0 de -LX2 st ≤0 LY0 de -LY0 st ≥0 LY0 de -LY2 st ≤0 LX2 de -LX0 st ≥0; LX2 de -LX2 st ≤0 LY2 de -LY0 st ≥0 <h2 style=";text-align:left;direction:ltr">LY2<h2 style=";text-align:left;direction:ltr"> de <h2 style=";text-align:left;direction:ltr"> -LY2<h2 style=";text-align:left;direction:ltr"> st <h2 style=";text-align:left;direction:ltr"> ≤0 When the above inequalities are true at the same time, the record base number is completely contained in the number set of the module base The determination result based on the identification point determines whether the area of ​​the mold bottom plate meets the preset conditions. The area determination results of the mold bottom plate are obtained, including: According to the collection Combined with the set BDZB, calculate whether the area of ​​the mold base meets the preset conditions The calculation formula is: when When , it means that the preset conditions are met. , it means that the preset conditions are not met; The method of determining the number of digits in the module bottom plate based on the area determination result of the mold bottom plate and the inclusion relationship result to obtain the number determination result includes: when When the record is MBN K :{M k ,B k ,N1 k ,N2 k ,N3 k ,N4 k }, Among them, M is the mold in the set SBZB m The subscript in the SBZB is the mold base plate. m The subscript in N1 is the number 1 of the mold base in the set SBZB m The subscript in N2 is the number 2 of the mold base in the set SBZB m The subscript in N3 is the number 3 of the mold base in the set SBZB m The subscript in N4 is the number 4 of the mold base in the set SBZB m The subscript in The method of sorting the numbers on the mold base plate based on the number determination result to obtain the final mold number includes: Traversing the MBN collection K , will collect SBZB m In Sort in ascending order and put Combined into digital output CODE.

6. The visual recognition method according to claim 5, characterized in that: The step of judging whether the target enters or leaves the workstation based on the final model number includes: When a valid CODE is identified, it is recorded as the mold entering the station; When the identified CODE disappears or is updated, it is recorded as the mold leaving the station.

7. The visual recognition method according to claim 6, characterized in that: Also includes: When the mold is recorded as entering the workstation, the entry time is recorded; When it is recorded that the mold leaves the workstation, the departure time is recorded; According to the recorded entry time and the recorded exit time, the residence time of the mold in the workstation is obtained.

8. A visual recognition device, characterized in that: include: The calibration module is used to calibrate the ideal identification position of each station and configure the floating coefficient to obtain the calibration data; The recognition module is used to decode the video frames collected by the camera into pictures and input them into the neural network to obtain target recognition data; A judgment module is used to judge whether the target enters or leaves the workstation based on the calibration data and the target recognition data.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the visual recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the visual recognition method according to any one of claims 1 to 7 when executed.

Images