Region frame recognition method, system and terminal based on canvas under react framework

Abstract

The application belongs to the technical field of computer software application, and discloses a method, system and terminal for realizing region frame recognition based on a canvas under a react framework, which comprises the following steps: in a page, EasyWarmPlayer is used to read a video stream pushed by a background to play camera playing content and display; a layer of canvas is overlaid on a video player, and different operations are realized according to a region type selected by a user, including three modes of rectangular drawing, elliptical (circular) drawing and polygon drawing, region drawing is completed, and drawing point coordinates are obtained; the coordinates are automatically converted into a unified resolution size according to a camera playing region, and then the coordinate data is transmitted to a background storage through an interface and subjected to algorithm recognition; and the region algorithm frame is modified and deleted. The application can realize that multiple algorithm frame regions can be drawn under each camera, region blocks can be drawn according to user mouse habits, and different algorithm regions can be distinguished according to region colors.

Description

Technical Field

[0001] This invention relates to the field of computer software application technology, and in particular to a method, system, and terminal for region drawing frame recognition based on canvas under the React framework. Background Technology

[0002] Currently, camera neural networks have a massive scale in China; and with the rapid development of computer neural networks, algorithm recognition applications are also rapidly becoming widespread across various applications; the combination of the two is a trend. Under the same camera scene, recognizing different algorithms based on different regions has become a very meaningful task. However, similar applications are very rare in the React framework, and most can only draw a single algorithm bounding box region for a single camera, unable to draw multiple algorithm bounding boxes for each camera, greatly reducing functionality for users. Therefore, this invention is a solution to this pain point. This invention can draw graphics in the same scene, obtain the coordinates of the drawn points, convert them to a uniform resolution, and then send them to the backend algorithm for recognition.

[0003] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0004] (1) There are few applications of algorithm box drawing under the React front-end application framework in the existing technology.

[0005] (2) Most technologies can only draw a single algorithm bounding box area based on a single camera, and cannot draw multiple algorithm bounding box areas for each camera, which greatly reduces the functionality for users. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method, system, and terminal for region frame recognition based on canvas under the React framework.

[0007] This invention is implemented as follows: a method for region frame recognition based on canvas within the React framework, comprising:

[0008] Step 1: On the designated page, use EasyWarmPlayer to read the video stream (H264 / H265) pushed from the background, play the content played by the camera, and display it.

[0009] Step two, a layer of canvas is overlaid on the camera player (in this case, EasyWarmPlayer player), and different scenes are realized according to the user-selected drawing area type, each algorithm area box corresponds to a canvas object, and the present application provides three drawing functions, which are rectangular drawing, elliptical (circular) drawing and polygon drawing;

[0010] Step three, modification and deletion are made to the area algorithm box.

[0011] Further, the specific process of rectangular drawing in step two is as follows:

[0012] (1.1) the user selects the rectangular drawing tool, the background code mechanism selects the drawing tool selected by the user, and the user confirms the starting point on the canvas according to the mouse;

[0013] (1.2) the drawing area is obtained by moving the mouse of the user, and it is judged whether the mouse of the user is released;

[0014] (1.3) when the user does not release the mouse, the react listening mechanism is used to draw a rectangle on the canvas according to the starting point and the coordinates of the current mouse movement, repeatedly fill and redraw in the process of mouse movement, and fill the corresponding color and line; after the user releases the mouse, the area drawing is completed and the drawing point coordinates are obtained, the coordinates are automatically converted into a unified resolution size (the resolution under the default (1920*1080) resolution) according to the camera playing area, and then the coordinate data is transmitted to the background storage through an interface and algorithm recognition is carried out.

[0015] Further, the specific process of rectangular drawing and circular drawing in step two is as follows:

[0016] (2.1) the user selects the elliptical (circular) and rectangular drawing tool, the background code mechanism selects the drawing tool selected by the user, and the user confirms the starting point on the canvas according to the mouse;

[0017] (2.2) the center of the circle is determined by the mouse click operation of the user through the react listening mechanism, and the drawing area is obtained by moving the mouse of the user to calculate the drawing radius;

[0018] (2.3) When the user finishes the area drawing after releasing the mouse, the center and radius will be automatically converted into a unified resolution size (default resolution under (1920*1080)) according to the camera playing area, and then the coordinate data is transmitted to the background storage through the interface. The backend receives the coordinates, intercepts the screenshot on the video stream using the coordinates, and converts it into Base64 bit format. The base64 picture is converted into a numpy array by calling opencv, and then the numpy array is put into the trained model. The model will establish a feature point vector and transmit the vector into the neural network for inference to obtain the result. Finally, the backend transmits the ID of the newly added region to the subsequent modification and deletion operations.

[0019] Further, the specific process of step two of polygon drawing is as follows:

[0020] (3.1) The user selects the polygon algorithm area tool, and the background code remembers the user's selection of the drawing tool. The user determines the starting point coordinates of the polygon algorithm area.

[0021] (3.2) Through user click listening operations on the canvas, the starting point is first determined by the click operation, and then each edge is drawn according to the movement of the mouse and the characteristics of the canvas. After determining the coordinates of each starting point, it is determined whether the coordinates are within ±5 pixels of the starting point.

[0022] (3.3) If not, continue to select point coordinates on the canvas to draw the area; if so, it is considered that the polygon algorithm area is completed at this time, and a closed figure is automatically filled with the selected color and drawn. Each selected point coordinate is converted into a unified resolution (default resolution under (1920*1080)), and then the coordinate data is transmitted to the background storage through the interface and subjected to algorithm recognition. Finally, the backend transmits the ID of the newly added region to the subsequent modification operation.

[0023] (3.4) According to the newly added region

[0024] Further, the specific process of step three of deletion is as follows:

[0025] (4.1) According to the ID returned by the backend when the newly added region is completed, the current drawing area canvas is obtained, and the canvas empty command is executed to delete the selected region.

[0026] When the algorithm area frame is drawn on the canvas, the user selects an initial point according to a mouse operation, and judges whether the algorithm area frame is a polygon or an ellipse (circle), if it is a polygon, the current point coordinates are obtained when the user clicks, and the point coordinates are compared with each coordinate in the polygon, to judge whether the point coordinates are in the interval of ±5 pixels of any polygon coordinate list; if yes, the point is used as a change point, other point coordinates remain unchanged, the mouse is moved to change the position of the point, and the canvas content is emptied and the figure is redrawn constantly by using the listening mechanism of react, and when the mouse is released, the drawing is completed again, and the modification is ended when the user releases the mouse; if no, the modification is ended;

[0027] if it is an ellipse (circle), the center point coordinates and the long radius or the short radius are used to judge whether the current coordinates are in the pixel interval of ±5 pixels; if yes, the coordinate closest to the coordinate point is determined as a moving point, the canvas content is emptied and the figure is redrawn constantly by using the listening mechanism of react; if no, the modification is ended;

[0028] When the modification is ended, the coordinate points are converted into a unified resolution (the resolution under the default (1920*1080) ), and are transmitted to the background to identify the algorithm area.

[0029] Another object of the present application is to provide a region frame recognition system based on a canvas under a react framework, which comprises:

[0030] An input module is used to read the video stream played by a camera and display the content by using an EasyWarmPlayer video stream player.

[0031] A region drawing module is used to realize different operations according to the drawing region type selected by the user, including three modes of rectangular drawing, elliptical (circular) drawing and polygon drawing.

[0032] A modification module is used to modify and delete the algorithm area frame.

[0033] Another object of the present application is to provide a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the region frame recognition method based on the canvas under the react framework.

[0034] Another object of the present application is to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method for recognizing a region frame based on a canvas implementation under a react framework.

[0035] Another object of the present application is to provide an information data processing terminal for a region frame recognition system based on a canvas implementation under a react framework.

[0036] In combination with the above technical solutions and the technical problems solved, the technical solutions to be protected by the present application have the following advantages and positive effects:

[0037] First, in view of the technical problems existing in the prior art and the difficulty in solving the problems, the technical solutions to be protected by the present application and the results and data obtained during the research and development process are combined in detail and deeply to analyze how the technical solutions solve the technical problems and bring some creative technical effects after solving the problems. The specific description is as follows:

[0038] (1) The region frame application under the react front-end framework is a good supplement.

[0039] (2) The traditional frame application component can only draw one frame, and this pain point is improved in depth. Originally, each camera can only draw one region frame, but now different graphics can be selected to draw multiple region frames. This greatly improves the user's work efficiency.

[0040] Secondly, from the perspective of the product as a whole, the technical solutions to be protected by the present application have the following technical effects and advantages, which are described in detail as follows:

[0041] The present application can draw multiple algorithm frame regions under each camera, draw region blocks according to the user's mouse habits, and distinguish different algorithm regions according to the region color.

[0042] Thirdly, as the creative auxiliary evidence of the claims of the present application, it is also reflected in the following important aspects:

[0043] (1) The expected income and commercial value of the technical solutions of the present application after transformation are as follows:

[0044] The present application provides an effective method for recognizing a region frame based on a canvas implementation under a react framework, which can draw multiple region blocks from one camera, greatly improving the user's functionality and having great commercial value.

[0045] (2) The technical solutions of the present application fill the technical gap in the industry at home and abroad:

[0046] The application realizes the drawing frame of the canvas area based on the react framework, and multiple algorithm frame areas can be drawn under each camera, which makes up for the blank that only a single area block can be drawn for a single camera in the industry at present.

[0047] (3) Does the technical solution of the application solve the technical problems that people have been eager to solve but have always failed to succeed?

[0048] Under the same camera scene, it is a very meaningful work to identify different algorithms according to different areas. However, there are few algorithm frame drawing applications under the react front-end application framework in the prior art. Moreover, only a single algorithm frame area can be drawn according to a single camera, and multiple algorithm frame areas cannot be drawn under each camera, which greatly reduces the functionality for users. The application proposes a canvas implementation area frame recognition method for algorithm frame drawing under the react front-end application framework, and solves the technical problem that multiple algorithm frame areas can be drawn under each camera.

[0049] (4) Does the technical solution of the application overcome the technical bias?

[0050] For algorithm frame drawing under the react front-end application framework, only a single algorithm frame area can be drawn according to a single camera. The canvas implementation area frame recognition method based on the react framework proposed by the application can draw graphics and obtain the drawing point coordinate conversion uniform resolution and then transmit it to the background algorithm recognition according to the user in the same scene, and multiple algorithm frame areas can be drawn under each camera, which breaks the traditional technical bias and greatly improves the practical value of the application. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is the flow chart of the canvas implementation area frame recognition method based on the react framework provided by the embodiment of the application;

[0052] Figure 2 is the flow chart of the rectangle drawing and circle drawing provided by the embodiment of the application;

[0053] Figure 3 is the flow chart of the polygon drawing provided by the embodiment of the application;

[0054] Figure 4 is the flow chart of the modification and deletion of the area algorithm frame provided by the embodiment of the application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below with examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0056] In order to make those skilled in the art fully understand how the present application is specifically implemented, this part is an explanation and description of the examples of the technical scheme of the claims.

[0057] As shown in Figure 1 The method for realizing region frame recognition based on the canvas under the react framework provided by the embodiment of the present application includes:

[0058] S101, in the page, the EasyWarmPlayer video stream player is used to read the video stream (H264 / H265) pushed by the background to play the content of the camera and display;

[0059] S102, a layer of canvas is overlaid on the video player, and different operations are realized according to the drawing region type selected by the user. Each algorithm region frame corresponds to a canvas object. The present application provides three drawing functions, which are rectangular drawing, elliptical (circular) drawing and polygon drawing.

[0060] S103, the region algorithm frame is modified and deleted.

[0061] As shown in Figure 2 The specific process of the rectangular drawing in the step S102 is as follows:

[0062] (1.1) the user selects the rectangular drawing tool, the background code mechanism selects the drawing tool selected by the user, and the user confirms the starting point on the canvas according to the mouse;

[0063] (1.2) the drawing region is obtained through the mouse movement of the user, and it is judged whether the mouse is released by the user;

[0064] (1.3) when the user does not release the mouse, the react listening mechanism is used to draw a rectangle on the canvas according to the coordinates of the starting point and the current mouse movement. The rectangle is repeatedly filled and redrawn in the process of mouse movement, and is filled with the corresponding color and line. When the user releases the mouse, the region drawing is completed and the drawing point coordinates are obtained. The coordinates are automatically converted into a unified resolution size (the resolution under the default (1920*1080)) according to the camera playing region, and then the coordinate data is transmitted to the background storage through an interface and is subjected to algorithm recognition.

[0065] The specific process of the rectangular drawing and the circular drawing in the step S102 is as follows:

[0066] (2.1) User selects ellipse (circle), rectangle drawing tool, background code mechanism user selects the drawing tool, user according to the mouse in the canvas to confirm the starting point;

[0067] (2.2) Through the listening mechanism of the mouse click operation of the user to determine the center of the circle, and through the user's mouse movement to get the drawing area to calculate the drawing radius;

[0068] (2.3) When the user releases the mouse, the area drawing is completed, and the center and radius will be automatically converted into a unified resolution size (default resolution under (1920*1080)) according to the camera playing area, and then the coordinate data is transmitted to the background storage through the interface and algorithm recognition.

[0069] As Figure 3 shown, the specific process of the polygon drawing in step S102 is:

[0070] (3.1) User selects the polygon algorithm area tool, and the background code remembers the user's selection of the drawing tool. The user determines the starting point coordinates of the polygon algorithm area.

[0071] (3.2) Through the user's click on the canvas, the starting point is determined first, and then each edge is drawn according to the movement of the mouse and the characteristics of the canvas. Each time a starting point coordinate is determined, it is judged whether the coordinate is a point within ±5 pixels of the starting point.

[0072] (3.3) If not, continue to select point coordinates on the canvas to draw the area. If so, it is considered that the polygon algorithm area is completed at this time, a closed figure is drawn with the selected color, and each selected point coordinate is converted into a unified resolution (default resolution under (1920*1080)), and then the coordinate data is transmitted to the background storage through the interface and algorithm recognition.

[0073] As Figure 4 shown, the specific process of step S103 is:

[0074] When the drawing is completed, the algorithm area frame is framed on the canvas, the user selects an initial point according to a mouse operation, it is judged whether the algorithm area frame is a polygon or an ellipse (a circle), if it is a polygon, the current point coordinates are obtained when the user clicks, the point coordinates are compared with each coordinate in the polygon, it is judged whether the point coordinates are in the interval of ±5 pixels of any polygon coordinate list; if yes, the point is used as a change point, other point coordinates are unchanged, the mouse is moved to change the position of the point, the canvas content is emptied and the figure is redrawn constantly by using the listening mechanism of react, when the mouse is released, the redrawing is completed, when the user releases the mouse to end the operation, the modification is ended; if no, it is ended;

[0075] if it is an ellipse (a circle), it is judged whether the current coordinates are in the pixel interval of ±5 pixels by using the center point coordinates and the long radius or the short radius; if yes, the coordinate closest to the coordinate point is determined and is used as a moving point, the canvas content is emptied and the figure is redrawn constantly by using the listening mechanism of react; if no, it is ended;

[0076] When the modification is ended, the listening of react is ended, the coordinate points are converted into a unified resolution (a resolution under the default (1920*1080)) and are transmitted to the background to identify the algorithm area.

[0077] The area frame recognition system based on the canvas under the react framework provided by the embodiment of the application comprises:

[0078] An input module is used to read the video stream played by a camera and display the content by using an EasyWarmPlayer video stream player.

[0079] An area drawing module is used to realize different operations according to the drawing area type selected by a user, including three modes of rectangle drawing, ellipse (circle) drawing and polygon drawing.

[0080] A modification module is used to modify and delete the algorithm area frame.

[0081] The embodiment of the application has achieved some positive effects in the development or use process, and indeed has great advantages compared with the prior art.

[0082] The area frame application under the react front-end framework is a good supplement. The pain point that the traditional frame application component can only draw one frame is improved, originally each camera can only draw one area frame, and now different figures can be selected to draw multiple area frames. The working efficiency of the user is greatly improved.

[0083] It should be noted that embodiments of the present application can be realized by hardware, software, or a combination of software and hardware. The hardware portion can be realized by a special logic; the software portion can be stored in a memory and executed by a proper instruction execution system, such as a microprocessor or a specially designed hardware. A person of ordinary skill in the art can understand that the above-mentioned apparatus and method can be realized by computer executable instructions and / or included in processor control codes, such as a carrier medium, such as a magnetic disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The apparatus of the present application and its modules can be realized by a hardware circuit, such as a very large scale integrated circuit or a gate array, a semiconductor, such as a logic chip, a transistor, or a programmable hardware device, such as a field programmable gate array, a programmable logic device, or the like, by software executed by various types of processors, or by a combination of the above-mentioned hardware circuit and software, such as firmware.

[0084] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement, and improvement within the technical range disclosed by the present application, and within the spirit and principle of the present application, should be included in the protection scope of the present application.

Claims

1. A method for recognizing a frame of a region based on a canvas under a react framework, characterized in that, The method for realizing area frame recognition based on the canvas of the react framework comprises the following steps: Step 1, in the page, the EasyWarmPlayer is used to read the video stream H264 / H265 pushed by the background to play the camera playing content and display; Step 2, a layer of canvas is overlaid on the video player, and an algorithm area frame is drawn according to the drawing area type selected by the user, each algorithm area frame corresponds to a canvas object, and the drawing area type comprises rectangular drawing, elliptical or circular drawing and polygon drawing; Step 3, the algorithm area frame is modified and deleted; The specific process of the polygon drawing in step 2 is as follows: (3.1) a polygon algorithm area drawing tool is selected, the background code remembers the selected drawing tool, and the user determines the starting point coordinates of the polygon algorithm area; (3.2) the starting point is determined through the click operation, and then each side frame is drawn according to the movement of the mouse and the characteristics of the canvas, and whether the coordinates of each starting point are within ±5 pixels from the starting point is judged; (3.3) if not, the point coordinates on the canvas are continuously selected to draw the area; if yes, the polygon algorithm area is completed, a closed figure is automatically filled with the selected color, each selected point coordinate is converted into a unified resolution, and the coordinate data is transmitted to the background storage through an interface and subjected to algorithm recognition; The specific process of step 3 is as follows: After the drawing is completed, the algorithm area frame is on the canvas, an initial point is selected according to the mouse, it is judged whether the algorithm area frame is a polygon or an ellipse or a circle, if it is a polygon, it is judged whether the current coordinates are within ±5 pixels; if yes, the current coordinates are changed as the change point, the other point coordinates remain unchanged, the mouse is moved to change the position of the current coordinates, the canvas content is continuously emptied and the figure is redrawn by using the listening mechanism of react, and when the mouse is released, the redrawing is completed, and the modification is ended when the user releases the mouse; if not, it is ended; if it is an ellipse or a circle, it is judged whether the current coordinates are within ±5 pixels by the center point coordinates and the long radius or the short radius; if yes, the coordinate closest to the coordinate point is determined as the moving point, the canvas content is continuously emptied and the figure is redrawn according to the listening mechanism of react; if not, it is ended; After the modification is ended, the listening mechanism of react is ended, the coordinate point is converted into a unified resolution and transmitted to the background to identify the algorithm area. 2.The method of claim 1, wherein the method is implemented based on a canvas of a react framework. The specific process of the rectangular drawing in step 2 is as follows: (1.1) a rectangular drawing tool is selected, the background code remembers the selected drawing tool, and the user confirms the starting point on the canvas according to the mouse; (1.2) the drawing area is obtained by moving the mouse; (1.3)Finally, it is judged whether the mouse is released, and when the mouse is not released, the listening mechanism of react is used to draw a rectangle on the canvas according to the coordinates of the starting point and the current mouse movement, repeatedly fill in and redraw during the movement of the mouse, and fill in the corresponding color and line; When the mouse is released, the area drawing is completed and the drawing point coordinates are obtained, the coordinates are automatically converted into a unified resolution size according to the camera playing area, and the coordinate data is transmitted to the background storage through an interface and algorithm recognition is performed. 3.The method of claim 1, wherein the method is implemented based on a canvas of a react framework. The specific process of the ellipse or circle drawing in step two is: (2.1) Selecting ellipse or circle drawing, the background code remembers the selected drawing tool, and the starting point on the canvas is confirmed according to the mouse; (2.2) The center of the circle is determined through the mouse click operation of the listening mechanism of react, and the drawing area is obtained through the mouse movement to calculate the drawing radius; (2.3) When the mouse is released, the area drawing is completed, and the center and radius are automatically converted into a unified resolution size according to the camera playing area, and the coordinate data is transmitted to the background storage through an interface and algorithm recognition is performed.

4. The method of claim 1, wherein the method is implemented based on a canvas of a react framework. The unified resolution size is 1920*1080.

5. A system for implementing the method for recognizing a picture frame in a region implemented by a canvas under a react framework according to any one of claims 1-4, characterized in that, The area frame recognition system based on the canvas under the react framework comprises: An input module for reading the video stream played by the camera by using EasyWarmPlayer to play the camera playing content and display; An area drawing module for realizing different operations according to the selected drawing area type, including three modes of rectangle drawing, ellipse or circle drawing, and polygon drawing; A modification module for modifying and deleting the area algorithm frame.

6. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the area frame recognition method based on the canvas under the react framework as claimed in any one of claims 1-4.

7. A computer readable storage medium storing a computer program, the computer program being executed by a processor to make the processor execute the steps of the area frame recognition method based on the canvas under the react framework as claimed in any one of claims 1-4.

8. An information data processing terminal, characterized by The information data processing terminal is used to realize the area frame recognition system based on the canvas under the react framework as claimed in claim 5.

Images