Electronic chess manual generation method and device, equipment, storage medium and program product

Abstract

The invention provides an electronic chess manual generation method and device, electronic equipment, a computer readable storage medium and a computer program product, and the method comprises the steps: collecting an image comprising a target chessboard, the target chessboard comprising a plurality of chess pieces; based on the image, the positions of a plurality of chess pieces included in the target chessboard are identified to obtain the position of each chess piece on the target chessboard, and based on the image, chess piece identification is performed on the plurality of chess pieces to obtain at least two target chess pieces in the plurality of chess pieces, and different target chess pieces belong to different camps; based on the at least two target chess pieces, classifying the plurality of chess pieces to obtain a camp to which each chess piece belongs; and generating a target electronic chess manual based on the position of each chess piece on the target chessboard and the camp to which each chess piece belongs. Therefore, the accuracy of generating the electronic chess manual can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technologies, and in particular, to a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for generating an electronic chess score sheet. Background Art

[0002] When generating an electronic chess score sheet in related technologies, most of them directly use a classification model trained based on common chess piece styles to classify the camps of chess pieces. However, such a method has too high a dependence on the styles, especially the colors, of chess pieces. If an uncommon color is encountered, the phenomenon of incorrect camp classification will occur, resulting in low accuracy when generating an electronic chess score sheet. Summary of the Invention

[0003] Embodiments of the present application provide a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for generating an electronic chess score sheet, which can improve the accuracy when generating an electronic chess score sheet.

[0004] The technical solution of the embodiments of the present application is implemented as follows:

[0005] Embodiments of the present application provide a method for generating an electronic chess score sheet, including:

[0006] Collecting an image including a target chessboard, where the target chessboard includes a plurality of chess pieces;

[0007] Based on the image, performing position recognition on the plurality of chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard, and based on the image, performing chess piece recognition on the plurality of chess pieces to obtain at least two target chess pieces among the plurality of chess pieces, where different target chess pieces belong to different camps;

[0008] Based on the at least two target chess pieces, classifying the plurality of chess pieces to obtain the camps to which the chess pieces belong;

[0009] Generating a target electronic chess score sheet based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong.

[0010] Embodiments of the present application provide an apparatus for generating an electronic chess score sheet, including:

[0011] A collecting module, configured to collect an image including a target chessboard, where the target chessboard includes a plurality of chess pieces;

[0012] An identification module, configured to perform position identification on the multiple chess pieces included in the target chessboard based on the image, obtain the positions of the chess pieces on the target chessboard, and perform chess piece identification on the multiple chess pieces based on the image, obtain at least two target chess pieces among the multiple chess pieces, and different target chess pieces belong to different camps;

[0013] A classification module, configured to classify the multiple chess pieces based on the at least two target chess pieces, and obtain the camps to which the chess pieces belong;

[0014] A generation module, configured to generate a target electronic chess score based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong.

[0015] In the above solution, the apparatus further includes a detection module, configured to detect the acquisition angle of the image to obtain a detection result; the identification module is further configured to, when the detection result indicates that the acquisition angle of the image belongs to an angle interval formed by a first angle and a second angle, perform position identification on the multiple chess pieces included in the target chessboard based on the image, and obtain the positions of the chess pieces on the target chessboard; wherein, the acquisition angle is used to indicate the angle formed by the connection line between the image acquisition device and the target chessboard with respect to the horizontal plane, the first angle is used to indicate the angle when the connection line is perpendicular to the horizontal plane, and the second angle is used to indicate the minimum angle at which each chess piece on the target chessboard can be acquired when the connection line is inclined with respect to the horizontal plane.

[0016] In the above solution, the apparatus further includes a first sending module, configured to generate a prompt message indicating an image acquisition error and send the prompt message to the terminal when the detection result indicates that the acquisition angle of the image does not belong to the angle interval.

[0017] In the above solution, the identification module is further configured to perform image recognition on the image to obtain multiple chessboard intersection points formed by multiple chessboard lines on the target chessboard and multiple chess pieces on the target chessboard; for each chess piece, determine the target chessboard intersection point where the chess piece is located, and determine the position of the corresponding chess piece on the target chessboard based on the position of the target chessboard intersection point on the target chessboard.

[0018] In the above solution, the device further includes a segmentation module, which is configured to perform foreground segmentation on the image to obtain a foreground image corresponding to the target chessboard; perform image recognition on the foreground image to obtain multiple edge lines of the target chessboard, and determine multiple vertices of the target chessboard based on the multiple edge lines; wherein the number of vertices of the target chessboard is the same as the number of edge lines; construct a coordinate system corresponding to the target chessboard, and in the coordinate system, correct the foreground image based on the coordinates of each vertex to obtain a corrected image; the recognition module 4552 is further configured to perform position recognition on the multiple chess pieces included in the target chessboard based on the corrected image to obtain the positions of the chess pieces on the target chessboard.

[0019] In the above solution, the recognition module is further configured to perform image recognition on the image to obtain the multiple chess pieces included in the target chessboard in the image, and perform chess piece recognition on each chess piece to obtain the role of the chess piece; select at least two target chess pieces from the multiple chess pieces based on the role of each chess piece.

[0020] In the above solution, the classification module is further configured to perform style recognition on the multiple chess pieces to obtain the style of each chess piece, and determine the style of each target chess piece from the styles of the chess pieces; for each target chess piece, match the style of the target chess piece with the styles of other chess pieces respectively to obtain multiple matching results, and determine the camp to which the other chess pieces belong based on the multiple matching results; wherein the other chess pieces are the chess pieces other than the at least two target chess pieces among the multiple chess pieces.

[0021] In the above solution, the generation module is further configured to obtain the role of each chess piece included in the target chessboard; generate a target electronic chess score based on the role of each chess piece, the position of each chess piece on the target chessboard, and the camp to which each chess piece belongs.

[0022] In the above solution, the generation module is further configured to obtain a target style corresponding to the camp to which each chess piece belongs, and generate electronic chess pieces corresponding to the role of each chess piece by using the target style; determine the position of each chess piece on the target chessboard as the position of the corresponding electronic chess piece on the target electronic chess score; generate the target electronic chess score based on each electronic chess piece and the position of each electronic chess piece on the target electronic chess score.

[0023] In the above solution, the device further includes a second sending module, which is configured to send the target electronic chess score to the terminal, so that the terminal displays the target electronic chess score and the game guidance information of at least one camp in the target electronic chess score.

[0024] In the above solution, the device is applied to a chess score recognition system, and the chess score recognition system includes a position detection model and a chess piece classification model; the recognition module is further configured to, through the position detection model, based on the image, perform position recognition on the multiple chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard; the recognition module is further configured to, through the chess piece classification model, based on the image, perform chess piece recognition on the multiple chess pieces to obtain at least two target chess pieces among the multiple chess pieces; the classification module is further configured to, through the chess piece classification model, based on the at least two target chess pieces, classify the multiple chess pieces to obtain the camps to which the chess pieces belong.

[0025] An embodiment of the present application provides an electronic device, including:

[0026] A memory for storing executable instructions;

[0027] A processor for, when executing the executable instructions stored in the memory, implementing the method for generating an electronic chess score provided in the embodiment of the present application.

[0028] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to implement the method for generating an electronic chess score provided in the embodiment of the present application when executed.

[0029] An embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for generating an electronic chess score provided in the embodiment of the present application.

[0030] The embodiment of the present application has the following beneficial effects:

[0031] First, based on the acquired image including the target chessboard, perform position recognition on the multiple chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard, and perform chess piece recognition on the multiple chess pieces based on the image to obtain at least two target chess pieces among the multiple chess pieces. Then, based on the at least two target chess pieces, classify the multiple chess pieces to obtain the camps to which the chess pieces belong, so as to generate a target electronic chess score based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong. In this way, by using at least two target chess pieces among the multiple chess pieces to classify the camps of the multiple chess pieces, compared with the method of classifying the camps of chess pieces based on the color of the chess pieces in the related art, the accuracy of camp classification is improved, and thus when generating a target electronic chess score based on the camps to which the chess pieces belong, the accuracy of generating the electronic chess score is improved. Description of the Drawings

[0032] Figure 1 is a schematic structural diagram of a system for generating an electronic chess score according to an embodiment of the present application;

[0033] Figure 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0034] Figure 3 is a schematic flowchart of a method for generating an electronic chess score according to an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of a chessboard line and a chessboard intersection point according to an embodiment of the present application;

[0036] Figure 5 is a schematic diagram of a foreground image and a corrected image according to an embodiment of the present application;

[0037] Figure 6 is a schematic diagram of an electronic chess score according to an embodiment of the present application;

[0038] Figure 7 is a schematic diagram of guidance information for a chess game according to an embodiment of the present application;

[0039] Figure 8 is a schematic diagram of the training process of a position detection model and a chess piece classification model according to an embodiment of the present application;

[0040] Figure 9 is a schematic flowchart of a method for generating an electronic chess score according to an embodiment of the present application;

[0041] Figure 10 is a schematic diagram of a photographing score recognition control and an image acquisition interface according to an embodiment of the present application;

[0042] Figure 11 is a schematic diagram of the process of cropping an image according to an embodiment of the present application;

[0043] Figure 12 is a schematic diagram of the processing process of a chessboard detection module according to an embodiment of the present application;

[0044] Figure 13 is a schematic diagram of the processing process of a chess piece position detection and classification module according to an embodiment of the present application;

[0045] Figure 14 is a technical architecture diagram of a Chinese chess photographing score recognition solution using composite computer vision technology according to an embodiment of the present application. Detailed Description of the Invention

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe this application in detail with reference to the accompanying drawings. The described embodiments should not be construed as limitations on this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0047] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0048] In the following description, the terms "first / second / third" are merely used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged in a specific order or sequence when permitted, so that the embodiments of this application described here can be implemented in an order other than that illustrated or described here.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0050] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are explained. The nouns and terms involved in the embodiments of this application are subject to the following explanations.

[0051] 1) Client: Also known as the user side, it refers to a program that provides local services for users corresponding to the server. Except for some applications that can only run locally, it is generally installed on ordinary client machines and needs to cooperate with the server to run. That is, there needs to be a corresponding server and service program in the network to provide corresponding services. In this way, a specific communication connection needs to be established between the client and the server side to ensure the normal operation of the application program. For example, an autonomous driving client (such as a map navigation client).

[0052] 2) Artificial Intelligence (AI): It is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machine to have the functions of perception, reasoning, and decision-making.

[0053] 3) In response to, which is used to represent the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more of the executed operations can be real-time or can have a set delay; without special instructions, there is no restriction on the execution order of multiple executed operations.

[0054] 4) The Yolo (You Only Look Once) model is an algorithm for object detection using a convolutional neural network, which uses a predefined prediction region method to complete object detection.

[0055] 5) The software development kit (SDK, Software Development Kit) is a collection of relevant documents, examples, and tools for assisting in the development of a certain type of software.

[0056] 6) A chess scorebook is a book or atlas that describes the basic techniques of a chess game and the moves in the opening, middle game, and endgame with diagrams and text.

[0057] 7) Metric learning, that is, distance metric learning (DML, Distance Metric Learning), which is also similarity learning. By performing feature transformation to obtain a feature subspace, and by using metric learning, similar objects are made closer in distance and different objects are made farther apart. That is to say, metric learning needs to obtain certain core features (characteristics) of the objects. For example, to distinguish two people, 2 eyes and 1 nose - these are commonalities, and willow-leaf-shaped eyebrows and cherry-like mouths - these are characteristics. Its goal is to make the distance between similar objects small and the distance between dissimilar objects large under the learned distance metric.

[0058] See Figure 1 , Figure 1 is a schematic diagram of the architecture of the electronic chess scorebook generation system provided by the embodiments of the present application. An electronic chess scorebook generation client 401 is set on the terminal (exemplarily showing the terminal 400), and is displayed on the display interface (exemplarily showing the image interface 401-1). The terminal 400 is connected to the server 200 through the network 300. Among them, the network 300 can be a wide area network or a local area network, or a combination of the two, and uses a wireless or wired link to implement data transmission.

[0059] Among them, the server 200 is used to collect an image including a target chessboard, where the target chessboard includes a plurality of chess pieces; based on the image, perform position recognition on the plurality of chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard, and based on the image, perform chess piece recognition on the plurality of chess pieces to obtain at least two target chess pieces among the plurality of chess pieces, and different target chess pieces belong to different camps; based on the at least two target chess pieces, classify the plurality of chess pieces to obtain the camps to which the chess pieces belong; based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong, generate a target electronic chess score; and send the target electronic chess score to the terminal 400;

[0060] The terminal 400 is used to receive the target electronic chess score sent by the server 200 and display the target electronic chess score.

[0061] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs, Content Deliver Network), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a set-top box, a smart voice interaction device, a smart home appliance, a virtual reality device, a vehicle-mounted terminal, an aircraft, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected through wired or wireless communication methods, and no limitation is made in the embodiments of the present application.

[0062] Next, the electronic device for implementing the method for generating an electronic chess score provided in the embodiments of the present application will be described. Refer to Figure 2 , Figure 2 is a schematic structural diagram of the electronic device provided in the embodiments of the present application. The electronic device may be a server or a terminal. Taking the server shown in Figure 1 as an example, Figure 2 the electronic device shown in includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the terminal 400 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 3 all kinds of buses are labeled as the bus system 440.

[0063] The processor 410 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0064] The user interface 430 includes one or more output devices 431 that enable the display of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0065] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. The memory 450 optionally includes one or more storage devices that are physically remote from the processor 410.

[0066] The memory 450 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0067] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0068] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0069] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.;

[0070] A presentation module 453 for enabling information to be displayed (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (such as a display screen, a speaker, etc.);

[0071] An input processing module 454 for detecting and translating one or more user inputs or interactions from one of one or more input devices 432.

[0072] In some embodiments, the device provided by the embodiments of the present application may be implemented in software. Figure 2 A generating device 455 for an electronic chess score stored in the memory 450 is shown, which may be software in the form of a program, a plugin, etc., including the following software modules: a collection module 4551, an identification module 4552, a classification module 4553, and a generating module 4554. These modules are logical, and thus can be arbitrarily combined or further split according to the functions to be implemented. The functions of each module will be described below.

[0073] In other embodiments, the device provided by the embodiments of the present application may be implemented in hardware. As an example, the generating device for an electronic chess score provided by the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the generating method for an electronic chess score provided by the embodiments of the present application. For example, a processor in the form of a hardware decoding processor may employ one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0074] In some embodiments, a terminal or a server may implement the generating method for an electronic chess score provided by the embodiments of the present application by running a computer program. For example, the computer program may be a native program or a software module in an operating system; it may be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP or a web browser APP; it may also be a small program, that is, a program that only needs to be downloaded into the browser environment to run; or it may be a small program that can be embedded into any APP. In short, the above computer program may be an application program, a module, or a plugin in any form.

[0075] Based on the above description of the electronic chess score generation system and electronic device provided in the embodiments of the present application, the following describes the electronic chess score generation method provided in the embodiments of the present application. In actual implementation, the electronic chess score generation method provided in the embodiments of the present application can be implemented independently by a terminal or a server, or jointly implemented by a terminal and a server, taking Figure 1 the server 200 in as an example to separately execute the electronic chess score generation method provided in the embodiments of the present invention for illustration. Refer to Figure 3 , Figure 3 which is a flowchart of the electronic chess score generation method provided in the embodiments of the present application, and will be described in combination with Figure 3 the steps shown.

[0076] Step 101, the server collects an image including a target chessboard, and the target chessboard includes a plurality of chess pieces.

[0077] In actual implementation, the server may receive an image acquisition request sent by the terminal, and thus, in response to the image acquisition request, collect an image including the target chessboard based on the image acquisition device, or the server receives an electronic chess score generation request carrying an image including the target chessboard sent by the terminal, and thus analyzes the electronic chess score generation request to obtain an image including the target chess score.

[0078] In actual implementation, after collecting an image including the target chessboard, the acquisition angle of the image is detected to obtain a detection result, and then based on the detection result, it is determined whether the image meets the generation conditions of the electronic chess score; when the detection result indicates that the image meets the generation conditions of the electronic chess score, an electronic chess score corresponding to the target chessboard is generated based on the image, and when the detection result indicates that the image does not meet the generation conditions of the electronic chess score, a prompt message for prompting to re-collect the image is sent to the terminal so that the terminal re-collects an image including the target chessboard.

[0079] It should be noted that whether the generation conditions of the chess score are met is related to the acquisition angle when collecting the image. When the detection result indicates that the acquisition angle of the image belongs to the angle range formed by the first angle and the second angle, it is determined that the image meets the generation conditions of the electronic chess score, and thus, based on the image, the positions of the plurality of chess pieces included in the target chessboard are recognized to obtain the positions of the chess pieces on the target chessboard; when the detection result indicates that the acquisition angle of the image does not belong to the angle range, a prompt message indicating an image acquisition error is generated and sent to the terminal.

[0080] Wherein, the acquisition angle is used to indicate the angle formed by the connection line between the image acquisition device and the target chessboard with respect to the horizontal plane, the first angle is used to indicate the angle when the connection line is perpendicular to the horizontal plane, and the second angle is used to indicate the minimum angle at which each chess piece on the target chessboard can be collected when the connection line is inclined with respect to the horizontal plane.

[0081] It should be noted that all the angles involved here are angles between 0 degrees and 90 degrees. When the connecting line is inclined with respect to the horizontal plane, an acute angle and an obtuse angle will be formed between the connecting line and the horizontal plane. The angles involved here are all the corresponding acute angles.

[0082] In some embodiments, when the detection result indicates that the acquisition angle of the image does not belong to the angle interval, while generating a prompt message for indicating an image acquisition error, the acquired image will also be recorded, so as to use the image as a sample for subsequent model training, that is, training the subsequent position detection model and the chess piece classification model based on the image, thereby improving the model training efficiency and the processing accuracy of the model.

[0083] It should be noted that when the acquisition angle of the image belongs to the angle interval formed by the first angle and the second angle, an electronic chess score corresponding to the target chessboard is generated based on the image. Compared with the related art in which the image can only be acquired from above the chessboard and then the electronic chess score corresponding to the chessboard is generated based on the acquired image, the present application can still accurately recognize the chess score without relying on a specific shooting angle and distance, greatly reducing the requirements for the chessboard photo and facilitating user operation.

[0084] Step 102: Based on the image, perform position recognition on multiple chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard, and based on the image, perform chess piece recognition on the multiple chess pieces to obtain at least two target chess pieces among the multiple chess pieces, and different target chess pieces belong to different camps.

[0085] In actual implementation, the process of performing position recognition on multiple chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard can be as follows: perform image recognition on the image to obtain multiple chessboard intersection points formed by multiple chessboard lines on the target chessboard and multiple chess pieces on the target chessboard; for each chess piece, determine the target chessboard intersection point where the chess piece is located, and based on the position of the target chessboard intersection point on the target chessboard, determine the position of the corresponding chess piece on the target chessboard. Exemplarily, see Figure 4 , Figure 4 is a schematic diagram of the chessboard lines and chessboard intersection points provided by an embodiment of the present application. Based on Figure 4 , the black dot indicated by 401 is a chessboard intersection point on the chessboard, and the line indicated by 402 is a chessboard line on the chessboard.

[0086] In some embodiments, after collecting an image of the target chessboard, it is also possible to perform foreground segmentation on the image to obtain a foreground image corresponding to the target chessboard; perform image recognition on the foreground image to obtain multiple edge lines of the target chessboard, and based on the multiple edge lines, determine multiple vertices of the target chessboard; where the number of vertices of the target chessboard is the same as the number of edge lines; construct a coordinate system corresponding to the target chessboard, and in the coordinate system corresponding to the target chessboard, based on the coordinates of each vertex, correct the foreground image to obtain a corrected image; thus, the process of performing position recognition on multiple chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard can be to perform position recognition on multiple chess pieces included in the target chessboard based on the corrected image to obtain the positions of the chess pieces on the target chessboard.

[0087] It should be noted that the collected image includes the target chessboard and other backgrounds. Therefore, performing foreground segmentation on the image to obtain a foreground image corresponding to the target chessboard means removing the backgrounds irrelevant to the target chessboard in the image; at the same time, the edge lines here can be, for example, the four long side lines around the chessboard, and the vertices can be the four vertices forming the chessboard.

[0088] In actual implementation, in the coordinate system corresponding to the target chessboard, the process of correcting the foreground image based on the coordinates of each vertex to obtain a corrected image can be to obtain the inclination angles of each edge with respect to the horizontal and vertical coordinate axes of the coordinate system; based on the inclination angles, adjust the coordinates of each vertex to obtain target coordinates; based on the target coordinates and the inclination angles, adjust the coordinates of each pixel point in the image to obtain a corrected image.

[0089] Exemplarily, see Figure 5 , Figure 5 are schematic diagrams of the foreground image and the corrected image provided by an embodiment of the present application. Based on Figure 5 , Figure 5 in which a indicated in is the image collected including the target chessboard, perform foreground segmentation on the image to obtain the foreground image as indicated by b in Figure 5 , and then correct the foreground image to obtain Figure 5 the corrected image as indicated by c in.

[0090] In actual implementation, the process of performing chess piece recognition on multiple chess pieces based on the image to obtain at least two target chess pieces among the multiple chess pieces can be to perform image recognition on the image to obtain multiple chess pieces included in the target chessboard in the image, and perform chess piece recognition on each chess piece to obtain the role of the chess piece; based on the role of each chess piece, select at least two target chess pieces from the multiple chess pieces.

[0091] It should be noted that the roles of the chess pieces can be used to identify the chess pieces. For example, they can be the general, knight, elephant, king, etc. The target chess pieces are set in advance from among multiple chess pieces. For example, they can be the general and the king. At the same time, the number of camps here is related to the types of chess pieces. For example, in Chinese chess, there will be two camps, and there are also two target chess pieces, namely the general and the king.

[0092] It should be noted that the identification of each chess piece here can be text recognition or shape recognition. When the chess piece carries text content, the identification of each chess piece is to perform text recognition on the text content carried by each chess piece, so as to obtain the role of the chess piece based on the recognition result. When the chess piece does not carry text content, the identification of each chess piece is to perform shape recognition on the shape of each chess piece, so as to obtain the role of the chess piece based on the recognition result. For example, in ordinary Chinese chess, the chess pieces carry text content, so the identification of each chess piece is to perform text recognition on the text content carried by each chess piece, while in chess, the chess pieces do not carry text content, so the identification of each chess piece is to perform shape recognition on the shape of each chess piece.

[0093] Step 103: Classify the multiple chess pieces based on at least two target chess pieces to obtain the camp to which each chess piece belongs.

[0094] In actual implementation, the process of classifying the multiple chess pieces based on at least two target chess pieces to obtain the camp to which each chess piece belongs can be as follows: perform style recognition on the multiple chess pieces to obtain the styles of each chess piece, and determine the styles of each target chess piece from the styles of each chess piece; for each target chess piece, match the style of the target chess piece with the styles of other chess pieces respectively to obtain multiple matching results, and determine the camp to which the other chess pieces belong based on the multiple matching results; where the other chess pieces are the chess pieces other than at least two target chess pieces among the multiple chess pieces.

[0095] It should be noted that the style of the chess piece can refer to the color, shape, etc. of the chess piece. Determining the camp to which the other chess pieces belong based on the multiple matching results can be to select target other chess pieces from the other chess pieces. Specifically, obtain the matching degree of each matching result, and for each matching result, when the matching degree of the matching result reaches the matching degree threshold, determine the other chess piece corresponding to the matching result as the target other chess piece, and determine the camp to which each target chess piece belongs as the camp to which the corresponding target other chess piece belongs.

[0096] It should be noted that the matching degree threshold here can also be pre-set, such as 90%. In this way, the chess pieces are classified into camps based on their similarity with the target chess pieces, that is, the distance between each chess piece and each target chess piece in the camp information dimension is compared, and the chess pieces with closer distances are classified into the same camp. Compared with the solution of directly classifying camps by color and style, the present application realizes the accurate distinction of camps of chess pieces in any color and style, avoids dependence on chess piece style and color, greatly broadens the usage scenarios, has a wider range of applicable scenarios, is more flexible, and has a higher accuracy rate.

[0097] Step 104, generating a target electronic chess record based on the position of each chess piece on the target chessboard and the camp to which each chess piece belongs.

[0098] In actual implementation, the process of generating a target electronic chess record based on the position of each chess piece on the target chessboard and the camp to which each chess piece belongs may be to obtain the role of each chess piece included in the target chessboard; and generate the target electronic chess record based on the role of each chess piece, the position of each chess piece on the target chessboard and the camp to which each chess piece belongs.

[0099] It should be noted that the process of generating the target electronic chess record based on the role of each chess piece, the position of each chess piece on the target chessboard, and the camp to which each chess piece belongs may be to obtain the target style corresponding to the camp to which each chess piece belongs, and use the target style to generate electronic chess pieces corresponding to the role of each chess piece; determine the position of each chess piece on the target chessboard as the position of the corresponding electronic chess piece on the target electronic chess record; and generate the target electronic chess record based on each electronic chess piece and the position of each electronic chess piece on the target electronic chess record. For example, see Figure 6 , Figure 6 is a schematic diagram of an electronic chess score provided in an embodiment of the present application, based on Figure 6 , based on the position of each chess piece on the target chessboard and the camp to which each chess piece belongs, generate Figure 6 The target electronic chess record shown.

[0100] In some embodiments, after generating a target electronic chess record based on the position of each chess piece on the target chessboard and the camp to which each chess piece belongs, the target electronic chess record can also be sent to the terminal so that the terminal displays the target electronic chess record and the game guidance information of at least one camp in the target electronic chess record. The game guidance information is used to indicate the next chess position of each camp. For example, see Figure 7 , Figure 7 is a schematic diagram of the game guidance information provided by the embodiment of the present application, based on Figure 7 , 701 and 702 indicate the game guidance information of the two camps, respectively, which are used to indicate the next chess position of the corresponding camp.

[0101] It should be noted that after the target electronic chess score is generated, the legality of the score surface of the target electronic chess score can be verified. Only when the verification result indicates that the score surface of the target electronic chess score is legal, will the target electronic chess score be sent to the terminal.

[0102] In some embodiments, the method for generating an electronic chess score provided in the embodiments of the present application can also be applied to a chess score recognition system, which includes a position detection model and a chess piece classification model. Therefore, the process of identifying the positions of multiple chess pieces included in the target chessboard based on an image to obtain the positions of each chess piece on the target chessboard can be: through the position detection model, based on the image, identify the positions of multiple chess pieces included in the target chessboard to obtain the positions of each chess piece on the target chessboard; the process of identifying multiple chess pieces based on the image to obtain at least two target chess pieces among the multiple chess pieces can be: through the chess piece classification model, based on the image, identify multiple chess pieces to obtain at least two target chess pieces among the multiple chess pieces; and the process of classifying multiple chess pieces based on at least two target chess pieces to obtain the camp to which each chess piece belongs can be: through the chess piece classification model, based on at least two target chess pieces, classify multiple chess pieces to obtain the camp to which each chess piece belongs. Among them, the position detection model can be a YOLO model, and the chess piece classification model can be a metric learning model.

[0103] In some embodiments, before generating an electronic chess score based on the chess score recognition system, it is also necessary to train the position detection model and the chess piece classification model. Specifically, refer to Figure 8 , Figure 8 is a schematic diagram of the training process of the position detection model and the chess piece classification model provided in the embodiments of the present application. Based on Figure 8 , the training process of the position detection model and the chess piece classification model provided in the embodiments of the present application is implemented by the following steps.

[0104] Step 201, the server obtains a training sample image including a chessboard; where the chess score includes multiple chess pieces, and the training sample image carries a first label and a second label. The first label is used to indicate the true position of each chess piece on the chessboard in the training sample image, and the second label is used to indicate the true camp to which each chess piece belongs.

[0105] In actual implementation, first, a large number of chessboard photos of different styles can be simulated through a game engine and some art materials, and random backgrounds and shooting angles are added. In this way, the diversity of training data is greatly expanded. Then, after the training sample image is generated, the first label and the second label are obtained by annotating the training sample image.

[0106] Step 202: Based on the training sample images, use the position detection model to identify the positions of multiple chess pieces on the chessboard, and obtain the positions of each chess piece on the chessboard.

[0107] In actual implementation, as described above, the position detection model can be a YOLO model, which reduces the requirements for the quality of the photos taken by the user and greatly simplifies the operation complexity of the user. At the same time, the user's fine-tuning results can be directly used as manually labeled data for the iterative training of the chessboard detection model. As the usage increases, the detection accuracy is continuously improved, and the detection accuracy rate is increased.

[0108] Step 203: Based on the training sample images, use the chess piece classification model to identify multiple chess pieces, obtain at least two target chess pieces among the multiple chess pieces, and then use the chess piece classification model to classify the multiple chess pieces based on the at least two target chess pieces, and obtain the camps to which each chess piece belongs.

[0109] In actual implementation, as described above, the chess piece classification model can be a metric learning model. In the related art, the model used for classifying the camps of chess pieces is only applicable to several fixed styles in the image dataset, and has poor robustness under different lighting environments and shooting angles. In more extreme cases, a certain color chess piece represents the red side in one set of chess pieces and the black side in another set of chess pieces, which makes the method of using a simple classification model to judge the camp completely infeasible. The metric learning model in this application enables it to output the distance between two chess pieces in the dimension of camp information when two chess piece images are input, so that there is a reliable index to measure whether two chess pieces belong to the same camp. In practical applications, after identifying the types of chess pieces, then identify the target chess pieces among them, such as the general and the king chess pieces. Then, compare the distance between each chess piece and these two chess pieces in the dimension of camp information in turn, and classify the chess pieces with closer distances into the same camp, thus bypassing the dependence on color and style for camp distinction. In this way, the applicable scenarios are wider, the flexibility is stronger, and the accuracy rate is higher.

[0110] Step 204: Obtain the first loss between the positions of each chess piece on the chessboard and the first label, and the second loss between the camps to which each chess piece belongs and the second label.

[0111] Step 205: Update the model parameters of the position detection model based on the first loss to obtain a trained position detection model, and update the chess piece classification model based on the second loss to obtain a trained chess piece classification model.

[0112] In some embodiments, the method for generating an electronic chess score provided in the embodiments of the present application can also be applied to a terminal. Specifically, see Figure 9 , Figure 9It is a schematic flowchart of the method for generating an electronic chess score provided by an embodiment of the present application. Based on Figure 9 , the method for generating an electronic chess score provided by an embodiment of the present application can be implemented by the following steps.

[0113] Step 301, the terminal responds to a trigger operation on the displayed photo recognition score control and displays an image acquisition interface.

[0114] It should be noted that an application program such as a client that supports the generation of an electronic chess score is set on the terminal. The client can be a game client, a photo-taking client, etc. When the user opens the application program on the terminal and the terminal runs the application program, the terminal presents the view interface corresponding to the client. Among them, a photo recognition score control is displayed in the view interface, so as to respond to a trigger operation on the displayed photo recognition score control and display an image acquisition interface.

[0115] Exemplarily, referring to Figure 10 , Figure 10 It is a schematic diagram of the photo recognition score control and the image acquisition interface provided by an embodiment of the present application. Based on Figure 10 , when the user opens the application program on the terminal and the terminal runs the application program, the terminal presents the view interface corresponding to the client as indicated by a in Figure 10 . In response to a trigger operation on the displayed photo recognition score control indicated by 1001 in a, an image acquisition interface as indicated by b in Figure 10 is displayed.

[0116] Step 302, in response to an image acquisition instruction triggered based on the image acquisition interface, display a target electronic chess score generated based on the acquired image including the target chessboard, and the game guidance information of at least one camp in the target electronic chess score; wherein, the target electronic chess score is generated based on the positions of the pieces on the target chessboard and the camps to which the pieces belong.

[0117] It should be noted that the target electronic chess score here is generated based on the positions of the pieces on the target chessboard and the camps to which the pieces belong. The positions of the pieces on the target chessboard are obtained by recognizing the positions of the multiple pieces included in the target chessboard based on the image. The camps to which the pieces belong are obtained by classifying the multiple pieces based on at least two target pieces among the multiple pieces, and different target pieces belong to different camps. The at least two target pieces are obtained by recognizing the pieces among the multiple pieces based on the image.

[0118] In actual implementation, the image acquisition interface may also display a confirmation control, so that the image acquisition instruction is triggered based on this confirmation control; after receiving the image acquisition instruction triggered by the image acquisition interface, the terminal may send the acquired image including the target chessboard to the server, so that the server can perform position recognition on multiple chess pieces included in the target chessboard based on the image, obtain the positions of each chess piece on the target chessboard, and perform chess piece recognition on multiple chess pieces based on the image, obtain at least two target chess pieces among the multiple chess pieces, and different target chess pieces belong to different camps; classify multiple chess pieces based on at least two target chess pieces to obtain the camp to which each chess piece belongs; generate a target electronic chess score based on the positions of each chess piece on the target chessboard and the camp to which each chess piece belongs, so that the terminal can receive the target electronic chess score generated based on the acquired image including the target chessboard returned by the server, and then display the target electronic chess score generated based on the acquired image including the target chessboard, and the game guidance information of at least one camp in the target electronic chess score;

[0119] Alternatively, after receiving the image acquisition instruction triggered by the image acquisition interface, the terminal can directly perform position recognition on multiple chess pieces included in the target chessboard based on the image, obtain the positions of each chess piece on the target chessboard, and perform chess piece recognition on multiple chess pieces based on the image, obtain at least two target chess pieces among the multiple chess pieces, and different target chess pieces belong to different camps; classify multiple chess pieces based on at least two target chess pieces to obtain the camp to which each chess piece belongs; generate a target electronic chess score based on the positions of each chess piece on the target chessboard and the camp to which each chess piece belongs, so as to display the target electronic chess score generated based on the acquired image including the target chessboard, and the game guidance information of at least one camp in the target electronic chess score.

[0120] Exemplarily, continue to refer to Figure 10 , based on Figure 10 , the image acquisition interface may also display a confirmation control indicated by 1002 in b as shown in Figure 10 , so that in response to the image acquisition instruction triggered based on the confirmation control indicated by 1002 in b as shown in Figure 10 , display the target electronic chess score generated based on the acquired image including the target chessboard and the game guidance information of at least one camp in the target electronic chess score as shown in c in Figure 10 .

[0121] In some embodiments, in addition to displaying the game guidance information, the displayed game guidance information can also be adjusted. For example, analyze the chess score based on a local database or a cloud database to obtain the game guidance information for each camp, or the analysis depth can also be adjusted, such as analyzing based on a normal depth or a precise depth; in addition, the game guidance information can also be hidden. Exemplarily, continue to refer to Figure 10 , based on Figure 10 , in addition to displaying the game guidance information, it can also be based on Figure 10 The content indicated by the dashed box 1003 in c of, adjust the displayed game guidance information, or the analysis depth can also be adjusted, or the game guidance information can also be hidden.

[0122] In some embodiments, in response to an image acquisition instruction triggered based on the image acquisition interface, the process of displaying the target electronic chess score generated based on the acquired image including the target chessboard and the game guidance information of at least one camp in the target electronic chess score can be: in response to an image acquisition instruction triggered based on the image acquisition interface, display the acquired image including the target chessboard; in response to a cropping operation on the image, display the cropped image; in response to a determination instruction for the cropped image, display the target electronic chess score generated based on the cropped image and the game guidance information of at least one camp in the target electronic chess score.

[0123] It should be noted that since the acquired image not only includes the target chessboard but also other backgrounds irrelevant to the target chessboard, after the image is acquired, the image can be cropped, and thus the target electronic chess score is generated based on the cropped image. In this way, by cropping the image, the recognition content of the image is reduced, the accuracy of image recognition is improved, that is, the accuracy of the generated target electronic chess score is improved.

[0124] Exemplarily, refer to Figure 11 , Figure 11 is a schematic diagram of the process of cropping an image provided by an embodiment of the present application. Based on Figure 11 , after the image indicated by b in is acquired as Figure 10 , crop the image through the cropping frame indicated by 1101 in Figure 11 to obtain the cropped image in the cropping frame indicated by 1101, and thus generate the target electronic chess score based on the cropped image.

[0125] Applying the above embodiments of the present application, first, based on the acquired image including the target chessboard, the positions of multiple chess pieces included in the target chessboard are recognized to obtain the positions of each chess piece on the target chessboard, and based on the image, the chess pieces are recognized to obtain at least two target chess pieces among the multiple chess pieces. Then, based on the at least two target chess pieces, the multiple chess pieces are classified to obtain the camps to which each chess piece belongs. Thus, based on the positions of each chess piece on the target chessboard and the camps to which each chess piece belongs, a target electronic chess score is generated. In this way, by using at least two target chess pieces among the multiple chess pieces to classify the camps of the multiple chess pieces, compared with the method of classifying the camps of chess pieces based on the colors of the chess pieces in the related art, the accuracy of camp classification is improved. Therefore, when generating the target electronic chess score based on the camps to which each chess piece belongs, the accuracy of generating the electronic chess score is improved.

[0126] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0127] Most of the chess score generation schemes in the related art are of two types. The first is the notation scheme based on chips or manual methods, that is, by implanting chips in the chess pieces to determine the positions of the chess pieces, or directly manually recording the chess score; the second is the score recognition scheme based on vision, that is, using a camera to capture the chessboard image and judging the types and positions of the chess pieces according to the image algorithm. However, for the first method above, whether it is the scheme using built-in chips or the manual notation scheme, it is more inclined to solve the problem of notation in the competition scenario. When dealing with the notation needs of general users, the hardware requirements or labor costs are too high, and the problem of cumbersome processes is particularly prominent and almost impossible to solve; for the second method above, directly classifying the camps through a classification model trained based on the chess pieces of common competition styles leads to too high a dependence on the styles of Chinese chess, especially the colors, such as red and black, red and green. If uncommon colors such as yellow and purple are encountered, serious camp classification errors will occur. Moreover, it is mostly used in competition scenarios or specific hardware, with high requirements for the shooting angle of the photo and the proportion of the chessboard in the photo, which is not convenient for users to operate manually and difficult to accurately locate.

[0128] Based on this, the embodiments of the present application provide a Chinese chess photo recognition and notation scheme using composite computer vision technology. In terms of the product function form, it is adjusted from originally being used for the official continuous recording of the entire offline game of players to being used for ordinary mobile phone users to take photos of the chessboard and convert the current situation into an electronic chess score for analyzing the chess score or online chess playing, expanding the applicable scenarios and simplifying the function requirements and operation steps;

[0129] Meanwhile, a chessboard recognition module is added. Compared with the original solution that is only applicable to the competition scenario where the camera view and the styles of chess pieces are relatively fixed, a dedicated chessboard detection model is additionally added, greatly reducing the requirements for chessboard photos and enabling accurate recognition of the chessboard position under any photographing conditions, which is convenient for users to operate;

[0130] In addition, instead of adopting the original solution of classifying the camps of chess pieces based on colors, a metric learning method is used. By training a model that can output the camp feature distances between chess pieces, the distances between each chess piece and the "general" chess piece representing the camp are compared, achieving accurate differentiation of the camps of chess pieces in any color and style, avoiding the dependence on the styles and colors of chess pieces, and extending the accurate differentiation of camps from several specific styles on the market to all styles, greatly broadening the usage scenarios.

[0131] Next, the technical solution of this application will be described from the product side. As Figure 10 shown, an application program such as a client that supports the generation of electronic chess scores is set on the terminal. When the user opens the application program on the terminal and the terminal runs the application program, the terminal presents the view interface corresponding to the client. Among them, a photo-taking score recognition control is displayed in the view interface, so that in response to the trigger operation on the displayed photo-taking score recognition control, an image acquisition interface is displayed. Then, in response to the image acquisition instruction triggered by the user, the server automatically detects the chessboard range and then returns it to the mobile phone for the user to adjust and confirm. Finally, the server differentiates the positions and camps of the chess pieces, generates an electronic chess score, and returns it to the mobile phone for the user to play chess or analyze.

[0132] Next, the technical solution of this application will be described from the technical side. Among them, the technical solution of this application mainly consists of the following four modules, namely, a chessboard detection module, a chess piece position detection and classification module, a chess score-related module, and a model retraining module.

[0133] For the chessboard detection module, refer to Figure 12 , Figure 12 which is a schematic diagram of the processing process of the chessboard detection module provided by the embodiment of this application. Based on Figure 12, first, using a game engine and some art materials, a large number of chessboard photos in different styles are simulated, accompanied by random backgrounds and shooting angles to obtain various chessboard photos, greatly expanding the diversity of training data. Supplementary with a small amount of manually taken and labeled data for model training. Then, using the YOLO model, compared with completely relying on manual operation by users, it greatly simplifies the operation complexity of users. At the same time, it reduces the requirements for the quality of the photos taken by users, greatly simplifies the operation complexity of users. Moreover, the fine-tuning results of users can be directly used as manually labeled data for iterative training of the chessboard detection model. As the usage increases, the detection accuracy is continuously improved, and the accuracy of detection is increased. In this way, using the object detection model, the position of the chessboard in the picture uploaded by the user is detected, and the result is cropped so that it can still accurately recognize the chess score without relying on a specific shooting angle and distance.

[0134] For the piece position detection and classification module, see Figure 13 , Figure 13 is a schematic diagram of the processing process of the piece position detection and classification module provided by the embodiment of the present application. Based on Figure 13 , similar to the chessboard detection module, first, using a game engine and some art materials, a large number of chessboard photos with different color font styles are simulated, greatly expanding the diversity of training data for model training. Then, using the metric learning model, it can output the distance between two pieces in the dimension of camp information when two piece pictures are input, so that there is a reliable index to measure whether two pieces belong to the same camp. Specifically, using the object detection model, that is, the chessboard detection module (position detection model), to detect the positions of each piece in the chessboard image. According to the detection results, the piece images at each position are cropped from the chessboard image and input into the image classification model, that is, the metric learning model (piece classification model). When judging the category of the piece, take out the "general" and "marshal" pieces among them, and compare them with all other pieces through the model trained by metric learning to obtain a distance in the dimension of piece camp. The pieces closer to the "general" piece are classified as the black side, and the pieces closer to the "marshal" piece are classified as the red side, ensuring that accurate camp classification can still be obtained without relying on the pieces having specific colors and styles. In this way, the applicable scenarios are wider, the flexibility is stronger, and the accuracy is higher.

[0135] For the chess score related module, after obtaining the coordinates and categories of each piece on the chessboard based on the chessboard detection module and the piece detection and classification module, the server will generate a logical chessboard according to this information and simultaneously verify the legality of the score surface. When the verification result indicates that the score surface is legal, it is sent to the client to be rendered on the user interface.

[0136] For the model retraining module, in actual implementation, if the recognition result is incorrect, the user can choose to report the error. At this time, the incorrect result and the picture will be saved on the server side. When a certain number accumulates, the retraining module can be enabled to further train the model, so as to improve the accuracy of the model.

[0137] In actual implementation, refer to Figure 14 , Figure 14 is the technical architecture diagram of the chess photo score recognition solution using the composite computer vision technology provided by the embodiments of the present application. Based on Figure 14 , first, the user manually takes a photo of the chess game to be recognized and uploads it to the server; then, after the server-side program obtains the chess game image, it forwards the picture to the chessboard detection module. After this module detects the position of the chessboard, it returns the detected chessboard coordinates to the client SDK; then, the client SDK hands the chessboard coordinates to the business layer of the chess and card game client. The business layer displays the range of the chessboard and uploads it to the server after fine-tuning by the client (optional); then, after the server-side program receives the confirmed chessboard range from the user, it crops the picture according to the chessboard range and forwards it to the chess piece detection and classification module. After this module recognizes the chess pieces, it compares the similarity of each chess piece with the two chess pieces of "general" to distinguish the camps, and then forwards the above information to the chess score-related module to generate the corresponding logical chessboard, and at the same time returns it to the client SDK; then, the client renders the game chessboard according to the logical chessboard, and finally completes the conversion of the real chess game image into an electronic chess game, which is convenient for subsequent online chess games and chess game analysis; in addition, for the pictures misjudged in the chessboard detection module and the chess piece position detection and classification module, the user can click to report an error and hand it to the model training server to further improve the recognition performance.

[0138] Applying the above embodiments of the present application, first, based on the collected image including the target chessboard, the positions of multiple chess pieces included in the target chessboard are recognized to obtain the positions of each chess piece on the target chessboard, and based on the image, multiple chess pieces are recognized to obtain at least two target chess pieces among the multiple chess pieces. Then, based on the at least two target chess pieces, the multiple chess pieces are classified to obtain the camps to which each chess piece belongs. Thus, based on the positions of each chess piece on the target chessboard and the camps to which each chess piece belongs, a target electronic chess score is generated. In this way, by using at least two target chess pieces among the multiple chess pieces to classify the camps of the multiple chess pieces, compared with the method of classifying the camps of chess pieces based on the color of the chess pieces in the related art, the accuracy of camp classification is improved, and thus when generating the target electronic chess score based on the camps to which each chess piece belongs, the accuracy of generating the electronic chess score is improved.

[0139] Next, continue to describe the exemplary structure of the implementation of the electronic chess score generation device 455 provided by the embodiments of the present application as a software module. In some embodiments, such as Figure 2As shown, the software modules in the electronic chess score generation device 455 stored in the memory 450 may include:

[0140] An acquisition module 4551, configured to acquire an image including a target chessboard, where the target chessboard includes a plurality of chess pieces;

[0141] An identification module 4552, configured to perform position identification on the plurality of chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard, and perform chess piece identification on the plurality of chess pieces based on the image to obtain at least two target chess pieces among the plurality of chess pieces, where different target chess pieces belong to different camps;

[0142] A classification module 4553, configured to classify the plurality of chess pieces based on the at least two target chess pieces to obtain the camps to which the chess pieces belong;

[0143] A generation module 4554, configured to generate a target electronic chess score based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong.

[0144] In some embodiments, the device further includes a detection module, configured to detect the acquisition angle of the image to obtain a detection result; the identification module 4552 is further configured to, when the detection result indicates that the acquisition angle of the image belongs to an angle range formed by a first angle and a second angle, perform position identification on the plurality of chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard; where the acquisition angle is used to indicate the angle formed by the connection line between the image acquisition device and the target chessboard with respect to the horizontal plane, the first angle is used to indicate the angle when the connection line is perpendicular to the horizontal plane, and the second angle is used to indicate the minimum angle at which each chess piece on the target chessboard can be acquired when the connection line is inclined with respect to the horizontal plane.

[0145] In some embodiments, the device further includes a first sending module, configured to generate a prompt message for indicating an image acquisition error and send the prompt message to the terminal when the detection result indicates that the acquisition angle of the image does not belong to the angle range.

[0146] In some embodiments, the identification module 4552 is further configured to perform image recognition on the image to obtain a plurality of chessboard intersection points formed by a plurality of chessboard lines on the target chessboard and a plurality of chess pieces on the target chessboard; for each chess piece, determine the target chessboard intersection point where the chess piece is located, and determine the position of the corresponding chess piece on the target chessboard based on the position of the target chessboard intersection point on the target chessboard.

[0147] In some embodiments, the device further includes a segmentation module, which is configured to perform foreground segmentation on the image to obtain a foreground image corresponding to the target chessboard; perform image recognition on the foreground image to obtain multiple edge lines of the target chessboard, and determine multiple vertices of the target chessboard based on the multiple edge lines; wherein the number of vertices of the target chessboard is the same as the number of the edge lines; construct a coordinate system corresponding to the target chessboard, and in the coordinate system, correct the foreground image based on the coordinates of each vertex to obtain a corrected image; the recognition module 4552 is further configured to perform position recognition on the multiple chess pieces included in the target chessboard based on the corrected image to obtain the positions of the chess pieces on the target chessboard.

[0148] In some embodiments, the recognition module 4552 is further configured to perform image recognition on the image to obtain the multiple chess pieces included in the target chessboard in the image, and perform chess piece recognition on each chess piece to obtain the role of the chess piece; select at least two target chess pieces from the multiple chess pieces based on the role of each chess piece.

[0149] In some embodiments, the classification module 4553 is further configured to perform style recognition on the multiple chess pieces to obtain the style of each chess piece, and determine the style of each target chess piece from the styles of the chess pieces; for each target chess piece, match the style of the target chess piece with the styles of other chess pieces respectively to obtain multiple matching results, and determine the camp to which the other chess pieces belong based on the multiple matching results; wherein the other chess pieces are the chess pieces other than the at least two target chess pieces among the multiple chess pieces.

[0150] In some embodiments, the generation module 4554 is further configured to obtain the role of each chess piece included in the target chessboard; generate a target electronic chess score based on the role of each chess piece, the position of each chess piece on the target chessboard, and the camp to which each chess piece belongs.

[0151] In some embodiments, the generation module 4554 is further configured to obtain the target style corresponding to the camp to which each chess piece belongs, and use the target style to generate electronic chess pieces corresponding to the role of each chess piece; determine the position of each chess piece on the target chessboard as the position of the corresponding electronic chess piece on the target electronic chess score; generate the target electronic chess score based on each electronic chess piece and the position of each electronic chess piece on the target electronic chess score.

[0152] In some embodiments, the device further includes a second sending module, configured to send the target electronic chess score to a terminal, so that the terminal displays the target electronic chess score and the game guidance information of at least one camp in the target electronic chess score.

[0153] In some embodiments, the device is applied to a chess score recognition system, and the chess score recognition system includes a position detection model and a chess piece classification model; the recognition module 4552 is further configured to, through the position detection model and based on the image, identify the positions of the multiple chess pieces included in the target chessboard, so as to obtain the positions of the chess pieces on the target chessboard; the recognition module 4552 is further configured to, through the chess piece classification model and based on the image, identify the multiple chess pieces to obtain at least two target chess pieces among the multiple chess pieces; the classification module 4553 is further configured to, through the chess piece classification model and based on the at least two target chess pieces, classify the multiple chess pieces to obtain the camps to which the chess pieces belong.

[0154] An embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method for generating an electronic chess score in the above embodiments of the present application.

[0155] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, where the executable instructions, when executed by a processor, cause the processor to execute the method for generating an electronic chess score provided in the embodiments of the present application. For example, as Figure 3 shown in the method for generating an electronic chess score.

[0156] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a CD-ROM, etc.; it may also be various devices including one or any combination of the above memories.

[0157] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a stand-alone program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0158] As an example, the executable instructions may or may not correspond to a file in a file system, may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or, stored in multiple cooperating files (such as files that store one or more modules, subroutines, or portions of code).

[0159] As an example, the executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or, on multiple electronic devices distributed across multiple locations and interconnected by a communication network.

[0160] In summary, the embodiments of the present application have the following beneficial effects:

[0161] (1) By using at least two target chess pieces among the multiple chess pieces to classify the camps of the multiple chess pieces, compared with the method of classifying the camps of chess pieces based on the color of the chess pieces in the related art, the accuracy of camp classification is improved, and thus, when generating a target electronic chess score based on the camps to which each chess piece belongs, the accuracy of generating the electronic chess score is improved.

[0162] (2) Classifying the camps of chess pieces by the similarity to the target chess pieces, that is, comparing the distance between each chess piece and each target chess piece in the dimension of camp information, and classifying the chess pieces with closer distances into the same camp. Compared with the solution of directly classifying the camps by color and style, the present application realizes the accurate distinction of the camps of chess pieces in any color and style, avoids the dependence on the style and color of the chess pieces, greatly broadens the usage scenarios, has a wider applicable scenario, stronger flexibility, and higher accuracy.

[0163] (3) By cropping the image, the recognition content of the image is reduced, and the accuracy of image recognition is improved, that is, the accuracy of the generated target electronic chess score is improved.

[0164] (4) When the acquisition angle of the image belongs to the angular range formed by the first angle and the second angle, an electronic chess score corresponding to the target chessboard is generated based on the image. Compared with the related art where the image can only be acquired from above the chessboard, and thus the electronic chess score corresponding to the chessboard is generated based on the acquired image, the present application can still accurately identify the chess score without relying on a specific shooting angle and distance, greatly reducing the requirements for the chessboard photos and facilitating user operation.

[0165] It should be noted that in the embodiments of the present application, operations related to users, acquisition of data such as images including the target chessboard, etc. When the embodiments of the present application are applied to specific products or technologies, corresponding permissions or consents need to be obtained, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0166] The above are only the embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application are all included in the protection scope of the present application.

Claims

1. A method for generating an electronic chess score, characterized in that, the method includes: collecting an image including a target chessboard, the target chessboard including a plurality of chess pieces; based on the image, performing position recognition on the plurality of chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard, and based on the image, performing chess piece recognition on the plurality of chess pieces to obtain at least two target chess pieces among the plurality of chess pieces, and different target chess pieces belong to different camps; classifying the plurality of chess pieces based on the at least two target chess pieces to obtain the camps to which the chess pieces belong; generating a target electronic chess score based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong.

2. The method according to claim 1, characterized in that, after collecting the image including the target chessboard, the method further includes: detecting the acquisition angle of the image to obtain a detection result; the performing position recognition on the plurality of chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard includes: when the detection result indicates that the acquisition angle of the image belongs to the angle range formed by the first angle and the second angle, performing position recognition on the plurality of chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard; wherein, the acquisition angle is used to indicate the angle formed by the connection line between the image acquisition device and the target chessboard relative to the horizontal plane, the first angle is used to indicate the angle when the connection line is perpendicular to the horizontal plane, and the second angle is used to indicate the minimum angle at which each chess piece on the target chessboard can be collected when the connection line is inclined to the horizontal plane.

3. The method according to claim 2, characterized in that, the method further includes: when the detection result indicates that the acquisition angle of the image does not belong to the angle range, generating a prompt message for indicating an image acquisition error and sending the prompt message to the terminal.

4. The method according to claim 1, characterized in that, the performing position recognition on the plurality of chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard includes: performing image recognition on the image to obtain a plurality of chessboard intersection points formed by a plurality of chessboard lines on the target chessboard and a plurality of chess pieces on the target chessboard; for each chess piece, determining the target chessboard intersection point where the chess piece is located, and based on the position of the target chessboard intersection point on the target chessboard, determining the position of the corresponding chess piece on the target chessboard.

5. The method according to claim 1, characterized in that, after collecting the image including the target chessboard, the method further includes, performing foreground segmentation on the image to obtain a foreground image corresponding to the target chessboard; Perform image recognition on the foreground image to obtain multiple edge lines of the target chessboard, and based on the multiple edge lines, determine multiple vertices of the target chessboard; wherein, the number of vertices of the target chessboard is the same as the number of edge lines; Construct a coordinate system corresponding to the target chessboard, and in the coordinate system, correct the foreground image based on the coordinates of each vertex to obtain a corrected image; The position recognition of the multiple chess pieces included in the target chessboard based on the image to obtain the positions of the chess pieces on the target chessboard includes: Based on the corrected image, perform position recognition on the multiple chess pieces included in the target chessboard to obtain the positions of the chess pieces on the target chessboard.

6. The method according to claim 1, characterized in that, The chess piece recognition of the multiple chess pieces based on the image to obtain at least two target chess pieces among the multiple chess pieces includes: Perform image recognition on the image to obtain the multiple chess pieces included in the target chessboard in the image, and perform chess piece recognition on each chess piece to obtain the role of the chess piece; Based on the roles of the chess pieces, select at least two target chess pieces from the multiple chess pieces.

7. The method according to claim 1, characterized in that, The classification of the multiple chess pieces based on the at least two target chess pieces to obtain the camps to which the chess pieces belong includes: Perform style recognition on the multiple chess pieces to obtain the styles of the chess pieces, and determine the styles of the target chess pieces from the styles of the chess pieces; For each target chess piece, match the style of the target chess piece with the styles of other chess pieces respectively to obtain multiple matching results, and based on the multiple matching results, determine the camps to which the other chess pieces belong; wherein, the other chess pieces are the chess pieces other than the at least two target chess pieces among the multiple chess pieces.

8. The method according to claim 1, characterized in that, The generation of the target electronic chess score based on the positions of the chess pieces on the target chessboard and the camps to which the chess pieces belong includes: Obtain the roles of the chess pieces included in the target chessboard; Generate a target electronic chess score based on the roles of the chess pieces, the positions of the chess pieces on the target chessboard, and the camps to which the chess pieces belong.

9. The method according to claim 8, characterized in that, The generation of the target electronic chess score based on the roles of the chess pieces, the positions of the chess pieces on the target chessboard, and the camps to which the chess pieces belong includes: Obtain the target styles corresponding to the camps to which the chess pieces belong, and use the target styles to generate electronic chess pieces corresponding to the roles of the chess pieces; Determine the positions of the chess pieces on the target chessboard as the positions of the corresponding electronic chess pieces on the target electronic chess score; Generate the target electronic chess score based on the electronic chess pieces and the positions of the electronic chess pieces on the target electronic chess score.

10. The method according to claim 1, characterized in that, After generating the target electronic chess score based on the positions of the pieces on the target chessboard and the camps to which the pieces belong, the method further includes: Sending the target electronic chess score to a terminal so that the terminal displays the target electronic chess score and the game guidance information of at least one camp in the target electronic chess score.

11. The method according to claim 1, wherein, the method is applied to a chess score recognition system, and the chess score recognition system includes a position detection model and a piece classification model; The step of performing position recognition on the multiple pieces included in the target chessboard based on the image to obtain the positions of the pieces on the target chessboard includes: Using the position detection model to perform position recognition on the multiple pieces included in the target chessboard based on the image to obtain the positions of the pieces on the target chessboard; The step of performing piece recognition on the multiple pieces based on the image to obtain at least two target pieces among the multiple pieces includes: Using the piece classification model to perform piece recognition on the multiple pieces based on the image to obtain at least two target pieces among the multiple pieces; The step of classifying the multiple pieces based on the at least two target pieces to obtain the camps to which the pieces belong includes: Using the piece classification model to classify the multiple pieces based on the at least two target pieces to obtain the camps to which the pieces belong.

12. An apparatus for generating an electronic chess score, wherein, the apparatus includes: An acquisition module for acquiring an image including a target chessboard, and the target chessboard includes multiple pieces; An identification module for performing position recognition on the multiple pieces included in the target chessboard based on the image to obtain the positions of the pieces on the target chessboard, and performing piece recognition on the multiple pieces based on the image to obtain at least two target pieces among the multiple pieces, and different target pieces belong to different camps; A classification module for classifying the multiple pieces based on the at least two target pieces to obtain the camps to which the pieces belong; A generation module for generating a target electronic chess score based on the positions of the pieces on the target chessboard and the camps to which the pieces belong.

13. An electronic device, wherein, it includes: A memory for storing executable instructions; A processor for implementing the method for generating an electronic chess score according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, wherein, it stores executable instructions for causing a processor to implement the method for generating an electronic chess score according to any one of claims 1 to 11 when executed.

15. A computer program product including a computer program or instruction, wherein, when the computer program or instruction is executed by a processor, it implements the method for generating an electronic chess score according to any one of claims 1 to 11.

Images